Tomcat NIO线程模型与IO方式分析

您所在的位置:网站首页 tomcat的nio Tomcat NIO线程模型与IO方式分析

Tomcat NIO线程模型与IO方式分析

2023-03-09 20:16| 来源: 网络整理| 查看: 265

本文参考了深度解读Tomcat中的NIO模型 和Tomcat NIO 模型的实现两篇文章,在看本文之前要先看一下这两位大神的分析,写的很好。但笔者对上述文章中关于NioBlockingSelector和BlockPoller的部分理解有些不太顺畅,所以下面只针对这一部分按照自己的思路整理出了本文。

Tomcat nio读request body与写response都是阻塞的

即使用了NIO模式,在tomcat中读取request body和写response的时候,根据servlet规范需要使用从Request和Response两个类获取流来进行读写,而ServletInputStream和 ServletOutputStream根据servlet规范是要求阻塞读写的。

比如在Request处理时,工作线程读请求line和header的时候是非阻塞的,而读request body是阻塞的。而由于accept的socket设置blocking false,所以要找到一个办法去让工作线程阻塞的去处理非阻塞的socket。

如何以阻塞的方式向非阻塞的socket进行读写

我们在org.apache.tomcat.util.net.NioEndpoint这个tomcat的NIO模式下连接和线程处理的核心组件中,可以找到Endponit初始化代码:

public void bind() throws Exception { initServerSocket(); //1、初始化Server Socket setStopLatch(new CountDownLatch(1)); // Initialize SSL if needed initialiseSsl(); selectorPool.open(getName()); //实例化shared selector,启动BlockPoller线程 }

直接看一下最后

//selectorPool.open public void open(String name) throws IOException { enabled = true; getSharedSelector(); //NioSelectorPool里的共享selector、区别于Poller里的selector if (shared) { blockingSelector = new NioBlockingSelector(); //实例化NioBlockingSelector blockingSelector.open(name, getSharedSelector()); //把共享selector传给NioBlockingSelector } } NioBlockingSelector.open() public void open(String name, Selector selector) { sharedSelector = selector; poller = new BlockPoller(); //实例化BlockPoller poller.selector = sharedSelector; poller.setDaemon(true); poller.setName(name + "-BlockPoller"); poller.start(); //启动BlockPoller线程 }

NioBlockingSelector 和 BlockPoller,前者提供读写方法、如果一次没读写完则阻塞在一个读写锁上等待通知就绪,后者内部是selector和轮询线程、负责epoll出来当前读写就绪的连接。当读写就绪时,就会打开连接上的读写锁(CountDownLatch实现),让阻塞在锁上的线程继续读写。接下来以写response为例,看一下这个过程。

这里我们可以从我们写的Servlet程序的response.getWriter().write()方法处用单步跟踪调试的办法一点一点的跟踪到NioBlockingSelector.write(),笔者也是这么试过的,但是也可以在“知道了答案的”情况下,偷懒直接通过在NioBlockingSelector.write()方法上打断点,通过观察线程栈的方法来了解这里的调用链路。断点打好之后,调用我们编写的servlet,在response.getWriter().write()写返回的时候,找到如下的调用栈:

"http-nio-8080-exec-2" #26 daemon prio=5 os_prio=0 tid=0x000000002964d000 nid=0x7390 at breakpoint[0x000000002a08e000] java.lang.Thread.State: RUNNABLE at org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:85) at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:152) at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.doWrite(NioEndpoint.java:1253) at org.apache.tomcat.util.net.SocketWrapperBase.doWrite(SocketWrapperBase.java:764) at org.apache.tomcat.util.net.SocketWrapperBase.flushBlocking(SocketWrapperBase.java:717) at org.apache.tomcat.util.net.SocketWrapperBase.flush(SocketWrapperBase.java:707) at org.apache.coyote.http11.Http11OutputBuffer$SocketOutputBuffer.end(Http11OutputBuffer.java:567) at org.apache.coyote.http11.filters.IdentityOutputFilter.end(IdentityOutputFilter.java:123) at org.apache.coyote.http11.Http11OutputBuffer.end(Http11OutputBuffer.java:234) at org.apache.coyote.http11.Http11Processor.finishResponse(Http11Processor.java:1162) at org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:389) at org.apache.coyote.Response.action(Response.java:209) at org.apache.catalina.connector.OutputBuffer.close(OutputBuffer.java:261) at org.apache.catalina.connector.Response.finishResponse(Response.java:443) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:374) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:374) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:888) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1597) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) - locked (a org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper) ava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745)

可以看到在response写了缓冲之后,针对response响应(注意不是socket连接)做了“end and flush”,将数据从缓冲区冲刷到socket,这个冲刷实际上就是NioSocketWrapper.doWrite() -> NioSelectorPool.write() -> NioBlockingSelector.write()这样一个调用链路。所以最后我们来看NioBlockingSelector.write()方法:

public int write(ByteBuffer buf, NioChannel socket, long writeTimeout) throws IOException { SelectionKey key = socket.getIOChannel().keyFor(socket.getSocketWrapper().getPoller().getSelector()); NioSocketWrapper att = (NioSocketWrapper) key.attachment(); int written = 0; boolean timedout = false; int keycount = 1; //assume we can write long time = System.currentTimeMillis(); //start the timeout timer while (!timedout && buf.hasRemaining()) { if (keycount > 0) { //only write if we were registered for a write int cnt = socket.write(buf); //write the data 写数据 if (cnt == -1) { throw new EOFException(); } written += cnt; if (cnt > 0) { time = System.currentTimeMillis(); //reset our timeout timer continue; //we successfully wrote, try again without a selector } } try { if (att.getWriteLatch() == null || att.getWriteLatch().getCount() == 0) { att.startWriteLatch(1); } poller.add(att, SelectionKey.OP_WRITE, reference); //注册到BlockPoller att.awaitWriteLatch(AbstractEndpoint.toTimeout(writeTimeout), TimeUnit.MILLISECONDS); //等待WriteLatch可写就绪 } catch (InterruptedException ignore) { // Ignore } } return written; }

上面有个awaitWriteLatch的操作,依靠writeLatch来阻塞在等待写就绪事件上,而写就绪事件已经通过BlockPoller.add注册到BlockPoller了,至此,如何使用一个非阻塞的socket模拟阻塞写的过程搞清楚了:

socket.write() -> 没写完,OP_WRITE注册到BlockPoller,线程阻塞等待awaitWriteLatch -> BlockPoller selector获得写就绪、writerLatch.countdown()通知 -> 线程继续写。

最后看一下tomcat9 的线程,可以看到跟8.5相比Poller只启动1个、就是那个ClientPoller线程,另外BlockPoller线程的作用上面也分析了。

Tomcat 9的线程

来自jstack的输出

Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.60-b23 mixed mode): "DestroyJavaVM" #27 prio=5 os_prio=0 tid=0x000000001dde2800 nid=0x5ce4 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "http-nio-8080-Acceptor" #26 daemon prio=5 os_prio=0 tid=0x000000001ddde000 nid=0x75d4 runnable [0x00000000219ae000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) - locked (a java.lang.Object) at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:469) at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:71) at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:106) at java.lang.Thread.run(Thread.java:745) "http-nio-8080-ClientPoller" #25 daemon prio=5 os_prio=0 tid=0x000000001dddf000 nid=0x3e6c runnable [0x00000000218ae000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked (a sun.nio.ch.Util$2) - locked (a java.util.Collections$UnmodifiableSet) - locked (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:711) at java.lang.Thread.run(Thread.java:745) "http-nio-8080-exec-3" #24 daemon prio=5 os_prio=0 tid=0x000000001bd46800 nid=0xfa4 waiting on condition [0x00000000217af000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) "http-nio-8080-exec-2" #23 daemon prio=5 os_prio=0 tid=0x000000001bd45800 nid=0x7584 waiting on condition [0x000000002145d000] "http-nio-8080-exec-1" #22 daemon prio=5 os_prio=0 tid=0x000000001bd43800 nid=0x5744 waiting on condition [0x000000002135f000] "http-nio-8080-BlockPoller" #21 daemon prio=5 os_prio=0 tid=0x000000001bd4a000 nid=0x7410 runnable [0x000000002003f000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278) at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked (a sun.nio.ch.Util$2) - locked (a java.util.Collections$UnmodifiableSet) - locked (a sun.nio.ch.WindowsSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.run(NioBlockingSelector.java:313) "container-0" #19 prio=5 os_prio=0 tid=0x000000001bd49800 nid=0x3558 waiting on condition [0x000000001fc3f000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.catalina.core.StandardServer.await(StandardServer.java:570) at org.springframework.boot.web.embedded.tomcat.TomcatWebServer$1.run(TomcatWebServer.java:180) "Catalina-utility-2" #18 prio=1 os_prio=-2 tid=0x000000001bd47000 nid=0x605c waiting on condition [0x000000001e92f000] ... "Catalina-utility-1" #17 prio=1 os_prio=-2 tid=0x000000001bd48800 nid=0x1454 waiting on condition [0x000000001e82e000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) "Service Thread" #10 daemon prio=9 os_prio=0 tid=0x000000001a12e000 nid=0x4b58 runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C1 CompilerThread3" #9 daemon prio=9 os_prio=2 tid=0x000000001a083800 nid=0x22e0 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C2 CompilerThread2" #8 daemon prio=9 os_prio=2 tid=0x000000001a074800 nid=0x5d90 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C2 CompilerThread1" #7 daemon prio=9 os_prio=2 tid=0x000000001a073000 nid=0x4d5c waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C2 CompilerThread0" #6 daemon prio=9 os_prio=2 tid=0x000000001a06d000 nid=0x4b0 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Attach Listener" #5 daemon prio=5 os_prio=2 tid=0x000000001a06a800 nid=0x1ed8 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Signal Dispatcher" #4 daemon prio=9 os_prio=2 tid=0x000000001a01f800 nid=0x5528 runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Finalizer" #3 daemon prio=8 os_prio=1 tid=0x0000000017f73800 nid=0x719c in Object.wait() [0x0000000019f7f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143) - locked (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) "Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x0000000002fa9000 nid=0x32dc in Object.wait() [0x0000000019e7f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157) - locked (a java.lang.ref.Reference$Lock) "VM Thread" os_prio=2 tid=0x0000000017f69000 nid=0x7264 runnable "GC task thread#0 (ParallelGC)" os_prio=0 tid=0x0000000002ecc000 nid=0x2234 runnable ... "GC task thread#7 (ParallelGC)" os_prio=0 tid=0x0000000002ed9800 nid=0x77e4 runnable "VM Periodic Task Thread" os_prio=2 tid=0x000000001a1a2000 nid=0x3b4 waiting on condition JNI global references: 1183


【本文地址】


今日新闻


推荐新闻


CopyRight 2018-2019 办公设备维修网 版权所有 豫ICP备15022753号-3