Spring SFTP : Unable to rename .writing file

I am using Spring SFTP integration to transfer the file and many time I got this error. It seems two threads are trying to transfer same file and conflict with each other

2020-08-03 08:31:55,766 INF [task-scheduler-8 ] o.s.i.ftp.session.FtpSession – File has been successfully transferred from: ./abc.ext.200803
2020-08-03 08:31:55,849 INF [task-scheduler-7 ] o.s.i.ftp.session.FtpSession – File has been successfully transferred from: ./abc.ext.200803
2020-08-03 08:31:55,850 INF [task-scheduler-7 ] .s.i.f.i.FtpInboundFileSynchronizer – Cannot rename ‘/local/download/abc.ext.200803.writing’ to local file ‘/local/download/abc.ext.200803’ after deleting. The local file may be busy in some other process.

Is there a way so both threads should not interfere with each other?

I am using following code –

@Bean
public SftpInboundFileSynchronizer ftpInboundFileSynchronizer() {
    isFTPSessionOK();
    SftpInboundFileSynchronizer fileSynchronizer = new SftpInboundFileSynchronizer(sftpSessionFactory());

    fileSynchronizer.setPreserveTimestamp(true);
    fileSynchronizer.setRemoteDirectory(remoteDirectory);
    fileSynchronizer.setDeleteRemoteFiles(false);

    fileSynchronizer.setFilter(new SFTPLastModifiedFileFilter(remoteFileFilter));
    return fileSynchronizer;
}

private boolean isFTPSessionOK() {
    try {
        SessionFactory<LsEntry> ftpSessionFactory = sftpSessionFactory();
        boolean open = ftpSessionFactory.getSession().isOpen();
        LOG.info("FTPSession is good ? " + open);
        return open;
    } catch (Exception e) {
        LOG.error("FTPSession is not good because of error : " + e);
    }
    return false;
}

@Bean
public SessionFactory<LsEntry> sftpSessionFactory() {
    DefaultSftpSessionFactory sf = new DefaultSftpSessionFactory();
    sf.setHost(server);
    sf.setPort(port);
    sf.setUser(username);
    sf.setPassword(password);
    sf.setAllowUnknownKeys(true);
    return new CachingSessionFactory<LsEntry>(sf);
}

@Bean
@InboundChannelAdapter(channel = "sftpChannel", poller = @Poller(fixedDelay = "${${project.name}.ftp.poller.delay:600000}", maxMessagesPerPoll = "1"))
public MessageSource<File> ftpMessageSource() {
    SftpInboundFileSynchronizingMessageSource source = new SftpInboundFileSynchronizingMessageSource(ftpInboundFileSynchronizer());
    source.setLocalDirectory(new File(localFtpDirectory));
    source.setAutoCreateLocalDirectory(true);       
    return source;
}

@Bean
@ServiceActivator(inputChannel = "sftpChannel")
public MessageHandler ftpHandler() {
    return new MessageHandler() {

        @Override
        public void handleMessage(Message<?> message) throws MessagingException {
            LOG.info("File '{}' is ready for reading after SFTP", message.getPayload());
        }
    };
}

Answer

You have only this for filtering:

fileSynchronizer.setFilter(new SFTPLastModifiedFileFilter(remoteFileFilter));

but what about a filter which is going to prevent duplicates on subsequent poll?

See AcceptOnceFileListFilter. And together with that SFTPLastModifiedFileFilter you should use a ChainFileListFilter.

See docs for more info:

https://docs.spring.io/spring-integration/docs/current/reference/html/sftp.html#sftp-inbound

https://docs.spring.io/spring-integration/docs/current/reference/html/file.html#file-reading

Advertisement

Answer