Solr4.7.2启动时的Index locked for write for core问题分析 |
您所在的位置：网站首页 › solr4 › Solr4.7.2启动时的Index locked for write for core问题分析 |
|
Solr在启动时，通过多线程的方式加载core，在加载完每个core的配置文件后，实例化了一个SolrCore，在SolrCore的构造函数中，会初始化index，Index locked for write for core的Exception就是在这个时候报出来的，Solr启动过程和SolrCore的构造函数可看之前的文章： http://blog.csdn.net/wenchanter/article/details/37603621 此处主要看下引发Index locked for write for core错误的代码： 首先看下SolrCore构造函数中的initIndex方法： void initIndex(boolean reload) throws IOException { // 着里面用dataDir生成了一个Directory，并设置了锁工厂 String indexDir = getNewIndexDir(); boolean indexExists = getDirectoryFactory().exists(indexDir); boolean firstTime; synchronized (SolrCore.class) { firstTime = dirs.add(getDirectoryFactory().normalize(indexDir)); } boolean removeLocks = solrConfig.unlockOnStartup; // solrconfig.xml中得indexReaderFactory,若不配置，也会默认StandardIndexReaderFactory initIndexReaderFactory(); if (indexExists && firstTime && !reload) { // 除了上面那个getNewIndexDir中的用dataDir创建的Directory设置了lockFactory，这里又用其返回的index路径创建了一个Directory，并设置了一遍lockFactory，真正判断是否加锁的时用这个Directory Directory dir = directoryFactory.get(indexDir, DirContext.DEFAULT, getSolrConfig().indexConfig.lockType); try { if (IndexWriter.isLocked(dir)) { if (removeLocks) { log.warn( logid + "WARNING: Solr index directory '{}' is locked. Unlocking...", indexDir); IndexWriter.unlock(dir); } else { log.error(logid + "Solr index directory '{}' is locked. Throwing exception", indexDir); throw new LockObtainFailedException( "Index locked for write for core " + name); } } } finally { directoryFactory.release(dir); } } // Create the index if it doesn't exist. if(!indexExists) { log.warn(logid+"Solr index directory '" + new File(indexDir) + "' doesn't exist." + " Creating new index..."); SolrIndexWriter writer = SolrIndexWriter.create("SolrCore.initIndex", indexDir, getDirectoryFactory(), true, getLatestSchema(), solrConfig.indexConfig, solrDelPolicy, codec); writer.close(); } } 可见是用IndexWriter.isLocked(dir)判断是否加锁，如果已经加了锁，则分为两种情况，一种是在solrconfig.xml中配置了unlockOnStartup，会尝试unlock，如果没有配置unlockStartup，则会抛出Index locked for write for core异常。这个被校验的dir是一个Directory，这个Directory是通过 Directory dir = directoryFactory .get(indexDir, DirContext . DEFAULT , getSolrConfig(). indexConfig . lockType );得到的，其中indexDir是上面取到的$solrhome/collection1/data/index索引存放的文件夹，directoryFactory此处是NRTCachingDirectoryFactory实例，看下其中的get方法： @Override public final Directory get(String path, DirContext dirContext, String rawLockType) throws IOException { String fullPath = normalize(path); synchronized (this) { if (closed) { throw new AlreadyClosedException("Already closed"); } final CacheValue cacheValue = byPathCache.get(fullPath); Directory directory = null; if (cacheValue != null) { directory = cacheValue.directory; } if (directory == null) { // 由于是NRTCachingDirectoryFactory，这里面直接new了一个NTRCachingDirectory，里面的FSDirectory.open(File)的方法根据不同的系统new不同的FSDirectory. directory = create(fullPath, dirContext); directory = rateLimit(directory); CacheValue newCacheValue = new CacheValue(fullPath, directory); // 这里设置锁工厂 injectLockFactory(directory, fullPath, rawLockType); byDirectoryCache.put(directory, newCacheValue); byPathCache.put(fullPath, newCacheValue); log.info("return new directory for " + fullPath); } else { cacheValue.refCnt++; log.debug("Reusing cached directory: {}", cacheValue); } return directory; } } get方法中的injectLockFactory设置了锁工厂，根据不同的属性设置不同的锁工厂，此处配置的是native，因此设置了一个NativeFSLockFactory： private static Directory injectLockFactory(Directory dir, String lockPath, String rawLockType) throws IOException { if (null == rawLockType) { // we default to "simple" for backwards compatibility log.warn("No lockType configured for " + dir + " assuming 'simple'"); rawLockType = "simple"; } final String lockType = rawLockType.toLowerCase(Locale.ROOT).trim(); if ("simple".equals(lockType)) { // multiple SimpleFSLockFactory instances should be OK dir.setLockFactory(new SimpleFSLockFactory(lockPath)); } else if ("native".equals(lockType)) { dir.setLockFactory(new NativeFSLockFactory(lockPath)); } else if ("single".equals(lockType)) { if (!(dir.getLockFactory() instanceof SingleInstanceLockFactory)) dir .setLockFactory(new SingleInstanceLockFactory()); } else if ("hdfs".equals(lockType)) { Directory del = dir; if (dir instanceof NRTCachingDirectory) { del = ((NRTCachingDirectory) del).getDelegate(); } if (del instanceof BlockDirectory) { del = ((BlockDirectory) del).getDirectory(); } if (!(del instanceof HdfsDirectory)) { throw new SolrException(ErrorCode.FORBIDDEN, "Directory: " + del.getClass().getName() + ", but hdfs lock factory can only be used with HdfsDirectory"); } dir.setLockFactory(new HdfsLockFactory(((HdfsDirectory)del).getHdfsDirPath(), ((HdfsDirectory)del).getConfiguration())); } else if ("none".equals(lockType)) { // Recipe for disaster log.error("CONFIGURATION WARNING: locks are disabled on " + dir); dir.setLockFactory(NoLockFactory.getNoLockFactory()); } else { throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Unrecognized lockType: " + rawLockType); } return dir; } 上面再new NativeFSLockFactory时，构造函数中调用父类FSLockFactory中的setLockDir(File lockDir)方法，这个方法只能调用一次，另外，在dir.setLockFactory时，也做了一些工作，看下一连串的方法调用： NTRCachingDirectory中的setLockFactory: @Override public void setLockFactory(LockFactory lf) throws IOException { // 这里是了NIOFSDirectory delegate.setLockFactory(lf); } 这里调用的是FSDirectory的setLockFactory: @Override public void setLockFactory(LockFactory lockFactory) throws IOException { super.setLockFactory(lockFactory); // for filesystem based LockFactory, delete the lockPrefix, if the locks are placed // in index dir. If no index dir is given, set ourselves if (lockFactory instanceof FSLockFactory) { final FSLockFactory lf = (FSLockFactory) lockFactory; final File dir = lf.getLockDir(); // if the lock factory has no lockDir set, use the this directory as lockDir if (dir == null) { lf.setLockDir(directory); lf.setLockPrefix(null); } else if (dir.getCanonicalPath().equals(directory.getCanonicalPath())) { lf.setLockPrefix(null); } } } super.setLockFactory是指父类BaseDirectory： @Override public void setLockFactory(LockFactory lockFactory) throws IOException { assert lockFactory != null; this.lockFactory = lockFactory; lockFactory.setLockPrefix(this.getLockID()); } 然后着里面的getLockID方法又使调用的FSDirectory中的,看看是不是可以发现LockID计算方法和String的hash算法是一样的： @Override public String getLockID() { ensureOpen(); String dirName; // name to be hashed try { dirName = directory.getCanonicalPath(); } catch (IOException e) { throw new RuntimeException(e.toString(), e); } int digest = 0; for(int charIDX=0;charIDX |
| CopyRight 2018-2019 办公设备维修网 版权所有 豫ICP备15022753号-3 |