Java中LocalCache本地缓存实现代码
前言
本次分享探讨java平台的本地缓存,是指占用JVM的heap区域来缓冲存储数据的缓存组件。
一、本地缓存应用场景
localcache有着极大的性能优势:
1. 单机情况下适当使用localcache会使应用的性能得到很大的提升。
2. 集群环境下对于敏感性要求不高的数据可以使用localcache,只配置简单的失效机制来保证数据的相对一致性。
哪些数据可以存储到本地缓存?
1.访问频繁的数据;
2.静态基础数据(长时间内不变的数据);
3.相对静态数据(短时间内不变的数据)。
二、java本地缓存标准
Java缓存新标准(javax.cache),这个标准由JSR107所提出,已经被包含在Java EE 7中。
特性:
1.原子操作,跟java.util.ConcurrentMap类似
2.从缓存中读取
3.写入缓存
4.缓存事件监听器
5.数据统计
6.包含所有隔离(ioslation)级别的事务
7.缓存注解(annotations)
8.保存定义key和值类型的泛型缓存
9.引用保存(只适用于堆缓存)和值保存定义
但目前应用不是很普遍。
三、java开源缓存框架
比较有名的本地缓存开源框架有:
1.EHCache
EHCache是一个纯java的在进程中的缓存,它具有以下特性:快速,简单,为Hibernate2.1充当可插入的缓存,最小的依赖性,全面的文档和测试。
BUG: 过期失效的缓存元素无法被GC掉,时间越长缓存越多,内存占用越大,导致内存泄漏的概率越大。
2.OSCache
OSCache有以下特点:缓存任何对象,你可以不受限制的缓存部分jsp页面或HTTP请求,任何java对象都可以缓存。拥有全面的API--OSCache API给你全面的程序来控制所有的OSCache特性。永久缓存--缓存能随意的写入硬盘,因此允许昂贵的创建(expensive-to-create)数据来保持缓存,甚至能让应用重启。支持集群--集群缓存数据能被单个的进行参数配置,不需要修改代码。缓存记录的过期--你可以有最大限度的控制缓存对象的过期,包括可插入式的刷新策略(如果默认性能不需要时)。
3.JCache
Java缓存新标准(javax.cache)
4.cache4j
cache4j是一个有简单API与实现快速的Java对象缓存。它的特性包括:在内存中进行缓存,设计用于多线程环境,两种实现:同步与阻塞,多种缓存清除策略:LFU, LRU, FIFO,可使用强引用。
5.ShiftOne
ShiftOne Java Object Cache是一个执行一系列严格的对象缓存策略的Java lib,就像一个轻量级的配置缓存工作状态的框架。
6.WhirlyCache
Whirlycache是一个快速的、可配置的、存在于内存中的对象的缓存。
四、LocalCache实现
1、LocalCache简介
LocalCache是一个精简版本地缓存组件,有以下特点:
1. 有容量上限maxCapacity;
2. 缓存达到容量上限时基于LRU策略来移除缓存元素;
3. 缓存对象的生命周期(缓存失效时间)由调用方决定;
4. 缓存对象失效后,将会有定时清理线程来清理掉,不会导致内存泄漏。
5. 性能比Ehcache稍强。
2、总体设计
LocalCache总体设计:
1. 缓存元素 CacheElement;
2. 缓存容器 LRULinkedHashMap;
3. 缓存接口 Cache;
4. 缓存组件实现 LocalCache。
3、详细设计
1. CacheElement设计
/** * 缓存元素 * */ public class CacheElement { private Object key; private Object value; private long createTime; private long lifeTime; private int hitCount; public CacheElement() { } public CacheElement(Object key ,Object value) { this.key = key; this.value = value; this.createTime = System.currentTimeMillis(); } public Object getKey() { return key; } public void setKey(Object key) { this.key = key; } public Object getValue() { hitCount++; return value; } public void setValue(Object value) { this.value = value; } public long getCreateTime() { return createTime; } public void setCreateTime(long createTime) { this.createTime = createTime; } public int getHitCount() { return hitCount; } public void setHitCount(int hitCount) { this.hitCount = hitCount; } public long getLifeTime() { return lifeTime; } public void setLifeTime(long lifeTime) { this.lifeTime = lifeTime; } public boolean isExpired() { boolean isExpired = System.currentTimeMillis() - getCreateTime() > getLifeTime(); return isExpired; } /* * (non-Javadoc) * @see java.lang.Object#toString() */ public String toString() { StringBuffer sb = new StringBuffer(); sb.append("[ key=").append(key).append(", isExpired=").append(isExpired()) .append(", lifeTime=").append(lifeTime).append(", createTime=").append(createTime) .append(", hitCount=").append(hitCount) .append(", value=").append(value).append(" ]"); return sb.toString(); } /* * (non-Javadoc) * @see java.lang.Object#hashCode() */ public final int hashCode(){ if(null == key){ return "".hashCode(); } return this.key.hashCode(); } /* * (non-Javadoc) * @see java.lang.Object#equals(java.lang.Object) */ public final boolean equals(Object object) { if ((object == null) || (!(object instanceof CacheElement))) { return false; } CacheElement element = (CacheElement) object; if ((this.key == null) || (element.getKey() == null)) { return false; } return this.key.equals(element.getKey()); } }
2. LRULinkedHashMap实现
import java.util.LinkedHashMap; import java.util.Set; import java.util.concurrent.locks.Lock; import java.util.concurrent.locks.ReentrantLock; /** * 实现 LRU策略的 LinkedHashMap * * @param <K> * @param <V> */ public class LRULinkedHashMap<K, V> extends LinkedHashMap<K, V> { protected static final long serialVersionUID = 2828675280716975892L; protected static final int DEFAULT_MAX_ENTRIES = 100; protected final int initialCapacity; protected final int maxCapacity; protected boolean enableRemoveEldestEntry = true;//是否允许自动移除比较旧的元素(添加元素时) protected static final float DEFAULT_LOAD_FACTOR = 0.8f; protected final Lock lock = new ReentrantLock(); public LRULinkedHashMap(int initialCapacity) { this(initialCapacity, DEFAULT_MAX_ENTRIES); } public LRULinkedHashMap(int initialCapacity ,int maxCapacity) { //set accessOrder=true, LRU super(initialCapacity, DEFAULT_LOAD_FACTOR, true); this.initialCapacity = initialCapacity; this.maxCapacity = maxCapacity; } /* * (non-Javadoc) * @see java.util.LinkedHashMap#removeEldestEntry(java.util.Map.Entry) */ protected boolean removeEldestEntry(java.util.Map.Entry<K, V> eldest) { return enableRemoveEldestEntry && ( size() > maxCapacity ); } /* * (non-Javadoc) * @see java.util.LinkedHashMap#get(java.lang.Object) */ public V get(Object key) { try { lock.lock(); return super.get(key); } finally { lock.unlock(); } } /* * (non-Javadoc) * @see java.util.HashMap#put(java.lang.Object, java.lang.Object) */ public V put(K key, V value) { try { lock.lock(); return super.put(key, value); } finally { lock.unlock(); } } /* * (non-Javadoc) * @see java.util.HashMap#remove(java.lang.Object) */ public V remove(Object key) { try { lock.lock(); return super.remove(key); } finally { lock.unlock(); } } /* * (non-Javadoc) * @see java.util.LinkedHashMap#clear() */ public void clear() { try { lock.lock(); super.clear(); } finally { lock.unlock(); } } /* * (non-Javadoc) * @see java.util.HashMap#keySet() */ public Set<K> keySet() { try { lock.lock(); return super.keySet(); } finally { lock.unlock(); } } public boolean isEnableRemoveEldestEntry() { return enableRemoveEldestEntry; } public void setEnableRemoveEldestEntry(boolean enableRemoveEldestEntry) { this.enableRemoveEldestEntry = enableRemoveEldestEntry; } public int getInitialCapacity() { return initialCapacity; } public int getMaxCapacity() { return maxCapacity; } }
3. Cache接口设计
/** * 缓存接口 * */ public interface Cache { /** * 获取缓存 * @param key * @return */ public <T> T getCache(Object key); /** * 缓存对象 * @param key * @param value * @param milliSecond 缓存生命周期(毫秒) */ public void putCache(Object key, Object value ,Long milliSecond); /** * 缓存容器中是否包含 key * @param key * @return */ public boolean containsKey(Object key); /** * 缓存列表大小 * @return */ public int getSize(); /** * 是否启用缓存 */ public boolean isEnabled(); /** * 启用 或 停止 * @param enable */ public void setEnabled(boolean enabled); /** * 移除所有缓存 */ public void invalidateCaches(); /** * 移除 指定key缓存 * @param key */ public void invalidateCache(Object key); }
4. LocalCache实现
import java.util.Date; import java.util.Iterator; import java.util.Random; import org.slf4j.Logger; import org.slf4j.LoggerFactory; /** * 本地缓存组件 */ public class LocalCache implements Cache{ private Logger logger = LoggerFactory.getLogger(this.getClass()); private LRULinkedHashMap<Object, CacheElement> cacheMap; protected boolean initFlag = false;//初始化标识 protected final long defaultLifeTime = 5 * 60 * 1000;//5分钟 protected boolean warnLongerLifeTime = false; protected final int DEFAULT_INITIAL_CAPACITY = 100; protected final int DEFAULT_MAX_CAPACITY = 100000; protected int initialCapacity = DEFAULT_INITIAL_CAPACITY;//初始化缓存容量 protected int maxCapacity = DEFAULT_MAX_CAPACITY;//最大缓存容量 protected int timeout = 20;//存取缓存操作响应超时时间(毫秒数) private boolean enabled = true; private Thread gcThread = null; private String lastGCInfo = null;//最后一次GC清理信息{ size, removeCount, time ,nowTime} private boolean logGCDetail = false;//记录gc清理细节 private boolean enableGC = true;//是否允许清理的缓存(添加元素时) private int gcMode = 0;//清理过期元素模式 { 0=迭代模式 ; 1=随机模式 } private int gcIntervalTime = 2 * 60 * 1000;//间隔时间(分钟) private boolean iterateScanAll = true;//是否迭代扫描全部 private float gcFactor = 0.5F;//清理百分比 private int maxIterateSize = DEFAULT_MAX_CAPACITY/2;//迭代模式下一次最大迭代数量 private volatile int iterateLastIndex = 0;//最后迭代下标 private int maxRandomTimes = 100;//随机模式下最大随机次数 protected final static Random random = new Random(); private static LocalCache instance = new LocalCache(); public static LocalCache getInstance() { return instance; } private LocalCache(){ this.init(); } protected synchronized void init() { if(initFlag){ logger.warn("init repeat."); return ; } this.initCache(); this.startGCDaemonThread(); initFlag = true; if(logger.isInfoEnabled()){ logger.info("init -- OK"); } } private void startGCDaemonThread(){ if(initFlag){ return ; } this.maxIterateSize = maxCapacity /2; try{ this.gcThread = new Thread() { public void run() { logger.info("[" + (Thread.currentThread().getName()) + "]start..."); //sleep try { Thread.sleep(getGcIntervalTime() < 30000 ? 30000 : getGcIntervalTime()); } catch (Exception e) { e.printStackTrace(); } while( true ){ //gc gc(); //sleep try { Thread.sleep(getGcIntervalTime() < 30000 ? 30000 : getGcIntervalTime()); } catch (Exception e) { e.printStackTrace(); } } } }; this.gcThread.setName("localCache-gcThread"); this.gcThread.setDaemon(true); this.gcThread.start(); if(logger.isInfoEnabled()){ logger.info("startGCDaemonThread -- OK"); } }catch(Exception e){ logger.error("[localCache gc]DaemonThread -- error: " + e.getMessage(), e); } } private void initCache(){ if(initFlag){ return ; } initialCapacity = (initialCapacity <= 0 ? DEFAULT_INITIAL_CAPACITY : initialCapacity); maxCapacity = (maxCapacity < initialCapacity ? DEFAULT_MAX_CAPACITY : maxCapacity); cacheMap = new LRULinkedHashMap<Object, CacheElement>(initialCapacity ,maxCapacity); if(logger.isInfoEnabled()){ logger.info("initCache -- OK"); } } /* * (non-Javadoc) */ @SuppressWarnings("unchecked") public <T> T getCache(Object key) { if(!isEnabled()){ return null; } long st = System.currentTimeMillis(); T objValue = null; CacheElement cacheObj = cacheMap.get(key); if (isExpiredCache(cacheObj)) { cacheMap.remove(key); }else { objValue = (T) (cacheObj == null ? null : cacheObj.getValue()); } long et = System.currentTimeMillis(); if((et - st)>timeout){ if(this.logger.isWarnEnabled()){ this.logger.warn("getCache_timeout_" + (et - st) + "_[" + key + "]"); } } if(logger.isDebugEnabled()){ String message = ("get( " + key + ") return: " + objValue); logger.debug(message); } return objValue; } /* * (non-Javadoc) */ public void putCache(Object key, Object value ,Long lifeTime) { if(!isEnabled()){ return; } Long st = System.currentTimeMillis(); lifeTime = (null == lifeTime ? defaultLifeTime : lifeTime); CacheElement cacheObj = new CacheElement(); cacheObj.setCreateTime(System.currentTimeMillis()); cacheObj.setLifeTime(lifeTime); cacheObj.setValue(value); cacheObj.setKey(key); cacheMap.put(key, cacheObj); long et = System.currentTimeMillis(); if((et - st)>timeout){ if(this.logger.isWarnEnabled()){ this.logger.warn("putCache_timeout_" + (et - st) + "_[" + key + "]"); } } if(logger.isDebugEnabled()){ String message = ("putCache( " + cacheObj + " ) , 耗时 " + (et - st) + "(毫秒)."); logger.debug(message); } if(lifeTime > defaultLifeTime && this.isWarnLongerLifeTime()){ if(logger.isWarnEnabled()){ String message = ("LifeTime[" + (lifeTime/1000) + "秒] too long for putCache(" + cacheObj + ")"); logger.warn(message); } } } /** * key 是否过期 * @param key * @return */ protected boolean isExpiredKey(Object key) { CacheElement cacheObj = cacheMap.get(key); return this.isExpiredCache(cacheObj); } /** * cacheObj 是否过期 * @param key * @return */ protected boolean isExpiredCache(CacheElement cacheObj) { if (cacheObj == null) { return false; } return cacheObj.isExpired(); } /* * (non-Javadoc) */ public void invalidateCaches(){ try{ cacheMap.clear(); }catch(Exception e){ e.printStackTrace(); } } /* * (non-Javadoc) */ public void invalidateCache(Object key){ try{ cacheMap.remove(key); }catch(Exception e){ e.printStackTrace(); } } /* * (non-Javadoc) */ public boolean containsKey(Object key) { return cacheMap.containsKey(key); } /* * (non-Javadoc) */ public int getSize() { return cacheMap.size(); } /* * (non-Javadoc) */ public Iterator<Object> getKeyIterator() { return cacheMap.keySet().iterator(); } /* * (non-Javadoc) */ public boolean isEnabled() { return this.enabled; } /* * (non-Javadoc) */ public void setEnabled(boolean enabled) { this.enabled = enabled; if(!this.enabled){ //清理缓存 this.invalidateCaches(); } } /** * 清理过期缓存 */ protected synchronized boolean gc(){ if(!isEnableGC()){ return false; } try{ iterateRemoveExpiredCache(); }catch(Exception e){ logger.error("gc() has error: " + e.getMessage(), e); } return true; } /** * 迭代模式 - 移除过期的 key * @param exceptKey */ private void iterateRemoveExpiredCache(){ long startTime = System.currentTimeMillis(); int size = cacheMap.size(); if(size ==0){ return; } int keyCount = 0; int removedCount = 0 ; int startIndex = 0; int endIndex = 0; try{ Object [] keys = cacheMap.keySet().toArray(); keyCount = keys.length; int maxIndex = keyCount -1 ; //初始化扫描下标 if(iterateScanAll){ startIndex = 0; endIndex = maxIndex; }else { int gcThreshold = this.getGcThreshold(); int iterateLen = gcThreshold > this.maxIterateSize ? this.maxIterateSize : gcThreshold; startIndex = this.iterateLastIndex; startIndex = ( (startIndex < 0 || startIndex > maxIndex) ? 0 : startIndex ); endIndex = (startIndex + iterateLen); endIndex = (endIndex > maxIndex ? maxIndex : endIndex); } //迭代清理 boolean flag = false; for(int i=startIndex; i<= endIndex; i++){ flag = this.removeExpiredKey(keys[i]); if(flag){ removedCount++; } } this.iterateLastIndex = endIndex; keys = null; }catch(Exception e){ logger.error("iterateRemoveExpiredCache -- 移除过期的 key时出现异常: " + e.getMessage(), e); } long endTime = System.currentTimeMillis(); StringBuffer sb = new StringBuffer(); sb.append("iterateRemoveExpiredCache [ size: ").append(size).append(", keyCount: ").append(keyCount) .append(", startIndex: ").append(startIndex).append(", endIndex: ").append(iterateLastIndex) .append(", removedCount: ").append(removedCount).append(", currentSize: ").append(this.cacheMap.size()) .append(", timeConsuming: ").append(endTime - startTime).append(", nowTime: ").append(new Date()) .append(" ]"); this.lastGCInfo = sb.toString(); if(logger.isInfoEnabled()){ logger.info("iterateRemoveExpiredCache -- 清理结果 -- "+ lastGCInfo); } } /** * 随机模式 - 移除过期的 key */ private void randomRemoveExpiredCache(){ long startTime = System.currentTimeMillis(); int size = cacheMap.size(); if(size ==0){ return; } int removedCount = 0 ; try{ Object [] keys = cacheMap.keySet().toArray(); int keyCount = keys.length; boolean removeFlag = false; int removeRandomTimes = this.getGcThreshold(); removeRandomTimes = ( removeRandomTimes > this.getMaxRandomTimes() ? this.getMaxRandomTimes() : removeRandomTimes ); while(removeRandomTimes-- > 0){ int index = random.nextInt(keyCount); boolean flag = this.removeExpiredKey(keys[index]); if(flag){ removeFlag = true; removedCount ++; } } //尝试 移除 首尾元素 if(!removeFlag){ this.removeExpiredKey(keys[0]); this.removeExpiredKey(keys[keyCount-1]); } keys=null; }catch(Exception e){ logger.error("randomRemoveExpiredCache -- 移除过期的 key时出现异常: " + e.getMessage(), e); } long endTime = System.currentTimeMillis(); StringBuffer sb = new StringBuffer(); sb.append("randomRemoveExpiredCache [ size: ").append(size).append(", removedCount: ").append(removedCount) .append(", currentSize: ").append(this.cacheMap.size()).append(", timeConsuming: ").append(endTime - startTime) .append(", nowTime: ").append(new Date()) .append(" ]"); this.lastGCInfo = sb.toString(); if(logger.isInfoEnabled()){ logger.info("randomRemoveExpiredCache -- 清理结果 -- "+ lastGCInfo); } } private boolean removeExpiredKey(Object key){ boolean flag = false; CacheElement cacheObj = null; if(null != key){ try{ cacheObj = cacheMap.get(key); boolean isExpiredCache = this.isExpiredCache(cacheObj); if(isExpiredCache){ cacheMap.remove(key); flag = true; } }catch(Exception e){ logger.error("removeExpired(" + key + ") -- error: " + e.getMessage(), e); } } if(!flag && logGCDetail){ this.logger.warn("removeExpiredKey(" + key + ") return [" + flag + "]--" + cacheObj); } return flag; } public int getInitialCapacity() { return initialCapacity; } public int getMaxCapacity() { return maxCapacity; } public int getGcMode() { return gcMode; } public void setGcMode(int gcMode) { this.gcMode = gcMode; } public int getGcIntervalTime() { return gcIntervalTime; } public void setGcIntervalTime(int gcIntervalTime) { this.gcIntervalTime = gcIntervalTime; } public boolean isEnableGC() { return enableGC; } public void setEnableGC(boolean enableGC) { this.enableGC = enableGC; } public boolean isIterateScanAll() { return iterateScanAll; } public void setIterateScanAll(boolean iterateScanAll) { this.iterateScanAll = iterateScanAll; } public float getGcFactor() { return gcFactor; } public void setGcFactor(float gcFactor) { this.gcFactor = gcFactor; } /** * gc 阀值 * @return */ public int getGcThreshold() { int threshold = (int)( this.cacheMap.getMaxCapacity() * gcFactor ); return threshold; } public String getLastGCInfo() { return lastGCInfo; } public void setLastGCInfo(String lastGCInfo) { this.lastGCInfo = lastGCInfo; } public boolean isLogGCDetail() { return logGCDetail; } public void setLogGCDetail(boolean logGCDetail) { this.logGCDetail = logGCDetail; } public int getTimeout() { return timeout; } public void setTimeout(int timeout) { this.timeout = timeout; } public int getMaxIterateSize() { return maxIterateSize; } public void setMaxIterateSize(int maxIterateSize) { this.maxIterateSize = maxIterateSize; } public int getMaxRandomTimes() { return maxRandomTimes; } public void setMaxRandomTimes(int maxRandomTimes) { this.maxRandomTimes = maxRandomTimes; } public boolean isInitFlag() { return initFlag; } public long getDefaultLifeTime() { return defaultLifeTime; } public boolean isWarnLongerLifeTime() { return warnLongerLifeTime; } public void setWarnLongerLifeTime(boolean warnLongerLifeTime) { this.warnLongerLifeTime = warnLongerLifeTime; } //======================== dynMaxCapacity ======================== private int dynMaxCapacity = maxCapacity; public int getDynMaxCapacity() { return dynMaxCapacity; } public void setDynMaxCapacity(int dynMaxCapacity) { this.dynMaxCapacity = dynMaxCapacity; } public void resetMaxCapacity(){ if(dynMaxCapacity > initialCapacity && dynMaxCapacity != maxCapacity){ if(logger.isInfoEnabled()){ logger.info("resetMaxCapacity( " + dynMaxCapacity + " ) start..."); } synchronized(cacheMap){ LRULinkedHashMap<Object, CacheElement> cacheMap0 = new LRULinkedHashMap<Object, CacheElement>(initialCapacity ,dynMaxCapacity); cacheMap.clear(); cacheMap = cacheMap0; this.maxCapacity = dynMaxCapacity; } if(logger.isInfoEnabled()){ logger.info("resetMaxCapacity( " + dynMaxCapacity + " ) OK."); } }else { if(logger.isWarnEnabled()){ logger.warn("resetMaxCapacity( " + dynMaxCapacity + " ) NO."); } } } //======================== showCacheElement ======================== private String showCacheKey; public String getShowCacheKey() { return showCacheKey; } public void setShowCacheKey(String showCacheKey) { this.showCacheKey = showCacheKey; } public Object showCacheElement(){ Object v = null; if(null != this.showCacheKey){ v = cacheMap.get(showCacheKey); } return v; } }
以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持我们。