Socket.close stuck for 15 minutes

I’ve developed my own MMORPG game client and server using SSLSockets. New connections get a dataoutput/input stream in their own threads. In my main thread loop, I have a method that goes through the connection map and closes connections (inactivity kicks, requested logouts, etc.).

My game server randomly hangs for 15 minutes so every 60 seconds, in another thread, I print out a stacktrace and how many loops the main thread has run.

main: [sun.misc.Unsafe.park(Native Method),
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175),
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836),
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870),
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199),
java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209),
java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285),
sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:863),
sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:735),
sun.security.ssl.SSLSocketImpl.sendAlert(SSLSocketImpl.java:2087),
sun.security.ssl.SSLSocketImpl.warning(SSLSocketImpl.java:1914),
sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1677),
sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1615),
com.jayavon.game.server.MyServer.processClientConnectionMapV2(MyServer.java:6427),
com.jayavon.game.server.MyServer.main(MyServer.java:708)]

From this I can tell that my server gets stuck when trying to close connections:

clientConnectionEntry.getValue().getSocket().close();

Here is the condensed method it is getting stuck running (which hangs the whole server):

private void processClientConnectionMapV2(boolean disconnectAll) {
    try {
        Iterator<Entry<String, ClientConnectionV2>> it = MyServer.ClientConnectionMapV2.entrySet().iterator();
        while (it.hasNext()) {
            Entry<String, ClientConnectionV2> clientConnectionEntry = it.next();
            
            if (clientConnectionEntry.getValue().isToDisconnect()
                    || disconnectAll){ //this connection should be disconnected
                if (!hasOutstandingBacklogMessagesV2(MyServer.ClientConnectionMapV2.get(clientConnectionEntry.getKey()))
                        || clientConnectionEntry.getValue().getDisconnectLoopsSkipped() > 4
                        || disconnectAll) {
                    MyServer.LOGGER.warn("----REMOVE CONNECTION----<processClientConnectionMapV2>----    " + clientConnectionEntry);
                    
                    //close the actual connection here
                    try {
                        clientConnectionEntry.getValue().getSocket().close();
                        clientConnectionEntry.getValue().getDataInputStream().close();
                        clientConnectionEntry.getValue().getDataOutputStream().close();
                        clientConnectionEntry.getValue().getCommandHandlerV2().loggedOut = true;
                        serverConnectionNumber--;
                    } catch (Exception e1) {
                        LOGGER.warn("unable to close in Server processClientConnectionsV2", e1);
                        LOGGER.warn("connection was already closed on client side");
                    }

                    //remove from iterator and ClientConnectionMapV2
                    it.remove();
                } else {
                    clientConnectionEntry.getValue().setDisconnectLoopsSkipped(clientConnectionEntry.getValue().getDisconnectLoopsSkipped() + 1);
                    MyServer.LOGGER.warn("----SKIP REMOVE CONNECTION----<processClientConnectionMapV2>----    " + clientConnectionEntry);
                }
            }
        }
    } catch (Exception e) {
        LOGGER.fatal("MAIN LOOP: processClientConnectionsV2: ", e);
    }
}

I’ve rewritten my connection code so much recently to try to fix the hangs and I have finally found the root cause – but I’m unsure how to address it.

Answer

For anyone who may find this here is how I solved it. I was unable to just rely on the setSoTimeout method as I wanted the ability to kick players ad hoc.

Thread closerThread = new Thread(){
    public void run(){
        //close the actual connection here
        try {
            Thread.currentThread().setName("closerThread-" + clientConnectionEntry.getKey() + "-" + tmpAccountName + "|" + tmpCharacterName);
            clientConnectionEntry.getValue().getCommandHandlerV2().outgoingMessageBacklog.clear();
            clientConnectionEntry.getValue().getCommandHandlerV2().loggedOut = true;
            serverConnectionNumber--;
            clientConnectionEntry.getValue().getSocket().close();
            clientConnectionEntry.getValue().getDataInputStream().close();
            clientConnectionEntry.getValue().getDataOutputStream().close();
        } catch (Exception e1) {
            LOGGER.warn("unable to close in Server processClientConnectionsV2", e1);
        }
    }
};
closerThread.start();

Advertisement

Answer