Bug report: FTP Upload

Post your questions about SoftEther VPN software here. Please answer questions if you can afford.
Post Reply
pklaas
Posts: 9
Joined: Sat Jun 07, 2014 10:47 pm

Bug report: FTP Upload

Post by pklaas » Sat Jun 07, 2014 10:58 pm

Hello, I have been along time userof Softhether vpn, great software.
Their has been a bug/problem for a long timeI noticed, I thought it would be fixed automaticly with updates, but after so long, still nothing.

The problem is when using this software as a vpn, surfing downloading and web uploading al goos fine.

But when uploading via ftp, normal ftp and ftp tls, it uploads some files not complete, it misses not in mb but rather a few thousand bites when tis a big file and few kb when its a small file.

I tested this with several servers, and played with settings, but the result moslty is the same, it does not upload some files compeltly.

For myself, its php files, exe files, html files. As these files I need to upload on regular bases to maintain a website.

Now only way it does it right and uploads all files 100% complete is using sftp, but many web hosts dont allow that as one needs shh/shell access.

I really hope you guys can replicate and fix this problem, this is the only draw back for me using this software, when I do the ftp uploads without using vpn it goos well.

I guess this bug might have stayed for so long unnoticed as perhaps not many people use it to upload to ftp, rather surf etc..

If anyone else wants to try this out pls do so, and report your problems perhaps it will get more attention, if you have a small hosting account use the vpn, ftp normally or with tls and upload for example installation files of joomla or soemthing, upload a big exe file and compare the sizes on your side and with the upload file you will notice the missing bytes kbs.

cheers.

dnobori
Posts: 228
Joined: Tue Mar 05, 2013 10:04 am

Re: Bug report: FTP Upload

Post by dnobori » Sun Jun 08, 2014 2:25 am

Technically, in the VPN tunnel, there are no difference between FTP and SFTP.
SoftEther VPN programs never discriminate against FTP from rest of any other protocols. So I suppose that your problem is caused not by FTP, but by other conditions such as network environment.

At this time, I cannot reproduce the problem. Any FTP-based file uploading completes as SFTP. There are no difference between FTP and SFTP.

To reproduce the problem, please post the following information:

1. Give us the example of the FTP server which your problem always occurs.
(To reproduce the problem, the FTP server should be a server which accepts uploading anonymously. Otherwise, no one except you could reproduce your problem.)
2. Give us the example of the large .exe file (or the uploaded URL) which you failed to upload in FTP (and also you succeeded to upload in SFTP in the same condition).
3. What is the file size of the large .exe file which always occurs your problem?
4. Which version of SoftEther VPN Server are you using? And where is the SoftEther VPN Server installed on?
5. Does SoftEther VPN Server use Local Bridge? or SecureNAT?
6. Are you using any anti-virus software or personal firewall? If yes, try again after uninstalling them.
7. When you provide the above information, the example of FTP server and the file must be matched to reproduce your problem. Otherwise, no one can reproduce the problem.

pklaas
Posts: 9
Joined: Sat Jun 07, 2014 10:47 pm

Re: Bug report: FTP Upload

Post by pklaas » Sun Jun 08, 2014 10:40 am

Hi dnobori,
thnx for the reply :)
point 1,2,3 no problem,
point 4, currently using the newest as wellas the version before,
and sofarI have hadthis problem with atleast 7 versions, as I always update in the hope the problem gets solved, I have beneusing this software when it was not opensource yet.

5. secure nat
6. allready tried in the past with and without, using different notebooks.
7. I can provide you ftp, but not anonymousas our host does not allow it.
perhaps you can give me your email, so i can email you logins of the ftp server.
Their I wll upload the same files without vpn and with, and you will be ble to see the difference in size,(difference in size is not much) butit breaks the files.

Its not only big files, also small files, butI used the big file as it shows the difference more clear, the smaller files is how I noticed the problem, when upgrading scripts, uploading wordpress or joomla, the installations or upgrades showed errors.

And the cause was incomplete uploads.
So if you could send me your email, I will send you ftp details, I tried finding anonymous ftp's but it seems hard to find were one can upload, iguess security risks.

Cheers.
ps. I was supriced too why sftp works but other ways dont,

dnobori
Posts: 228
Joined: Tue Mar 05, 2013 10:04 am

Re: Bug report: FTP Upload

Post by dnobori » Sun Jun 08, 2014 11:51 am

In the SecureNAT status window, "Kernel-mode NAT" is activated?

pklaas
Posts: 9
Joined: Sat Jun 07, 2014 10:47 pm

Re: Bug report: FTP Upload

Post by pklaas » Sun Jun 08, 2014 12:56 pm

hi, I just checked on the servers, on both of them it says no,
but I did not disable them in Virtual Hub Extended options.
How to still enable them ?
Is this you think causing this issue ?
Warm Regards,
Klaas

dnobori
Posts: 228
Joined: Tue Mar 05, 2013 10:04 am

Re: Bug report: FTP Upload

Post by dnobori » Sun Jun 08, 2014 2:23 pm

Kernel-mode Secure NAT improves the performance significantly.
Kernel-mode Secure NAT is automatically enabled when the both two conditions match:

1) The vpnserver process is running in the SYSTEM or root privileges.
2) The computer (or the Virtual Machine) has a network interface card which is permitted to activate the promiscuous mode. The network interface must be attached to the layer-2 Ethernet segment which has a DHCP server which designates the Default Gateway and DNS parameters to clients.

If not, kernel-mode secure NAT is unavailable, and user-mode secure NAT will be chosen automatically as an alternative.

Anyway, I am aware of the user-mode NAT performance problem on FTP file transmission.
The problem might be on a behavior of your target FTP server, especially the processing of TCP_RST or TCP_FIN packets. I am going to analyze the user-mode NAT code.


Please see the following instructions on the http://www.vpnusers.com/viewtopic.php?f=7&t=2790 page.
7. Are you using SecureNAT?
(If so, why don't you use the Local Bridge function instead?
The performance of SecureNAT is lower than Local Bridge, and it consumes
much of CPU time. You should not use SecureNAT except very limited situation.)
Please see http://www.softether.org/index.php?titl ... T_Function

pklaas
Posts: 9
Joined: Sat Jun 07, 2014 10:47 pm

Re: Bug report: FTP Upload

Post by pklaas » Sun Jun 08, 2014 3:25 pm

Hi,
the reason we using allways secure nat instead of localbridge function is, we use the vpn to bypas blocked websites/censorship.

People who work on our websites, need to have free access on the internet.

And the local bridge never managed to help us do this (we were able to connect but not use it to surf the internet for the users etc..), but when we enable the nat function it solved everything, so we allways used this.

And it worked excellent, we have several vps running these vpns's. Except the ftp thing offcourse. We even tried in the past changing the ftp server Pure-FTPD to ProFTPd as these are the 2 options to choose from (cpanel webhost controlpanel),it didnt make difference regarding the issue. (cpanel is most used webhost control panel).

We are willing to try to do the local bridge again, but can it do for what we are using it ? Because once we failed achieving that, we thought perhaps its not for that function. on your website their tutorials for different kind of vpn settings using bridge, but none seem to be explicite for what we are using it. It seems moe for cloud or lan settings. But maybe we looked it the wrong way.

But if the nat function could be fixed to handle ftp would eb great, since with nat function its eems to work well for our needs. (except ftp) and its easy setup :)

Warm Regards,
Klaas

pklaas
Posts: 9
Joined: Sat Jun 07, 2014 10:47 pm

Re: Bug report: FTP Upload

Post by pklaas » Sun Jun 08, 2014 3:38 pm

Also I was just testig on 1 vpn localbridge requires wincap and it says wincap not supported on version windows running. That is windows 2008 R2 Standaard.
Our other vpn running other windows. I could perhaps try, if you really reccomend trying that too for the function we using.

dnobori
Posts: 228
Joined: Tue Mar 05, 2013 10:04 am

Re: Bug report: FTP Upload

Post by dnobori » Sun Jun 08, 2014 4:56 pm

Local bridge needs promiscuous mode. Are you using a VPS or VM in a shared cloud? If so, you need to ask the administrator of the cloud server to allow you activate the promiscuous mode.

dnobori
Posts: 228
Joined: Tue Mar 05, 2013 10:04 am

Re: Bug report: FTP Upload

Post by dnobori » Mon Jun 09, 2014 4:11 am

Hi pklaas,

I uploaded SoftEther VPN 4.09 Build 9451 Beta (June 9, 2014).
This is a beta version.
It improves User-mode SecureNAT performance by modifying the processing of TCP_FIN packets. It should improve the performance of the FTP protocol.
http://www.softether.org/5-download/history

Please try this build, and let me know the result.

pklaas
Posts: 9
Joined: Sat Jun 07, 2014 10:47 pm

Re: Bug report: FTP Upload

Post by pklaas » Mon Jun 09, 2014 12:19 pm

hi dnobori,
Thnx for your work.
I have tested it, and it is a BIG difference with before, before it never succeeded even 1x,
Now its 100% complete roughly 2 out of 3 times. or 3 outof 10 times its not compplete.
But this is really betterthen before, as we have now options, alltough have to be carefull to double check everything.

I tested around 30x the file to get these results.
And that is for normal ftp transfer, I didnt tested the small files yet.

Transfering ftp with TLS is different story, that one 15 out of 15 were not 100% complete.

hope my data helps.

but still itsgood progress sincenormal ftp majority of time functions well, and localbridge needing promiscuous mode being an issue, as none of our vps providers either have that as option or want to do that.

Cheers,
Klaas

dnobori
Posts: 228
Joined: Tue Mar 05, 2013 10:04 am

Re: Bug report: FTP Upload

Post by dnobori » Mon Jun 09, 2014 1:03 pm

Hi,

About the large file, how many size is it?
When it succeed, how long does it take to be uploaded?

pklaas
Posts: 9
Joined: Sat Jun 07, 2014 10:47 pm

Re: Bug report: FTP Upload

Post by pklaas » Mon Jun 09, 2014 1:19 pm

hi, its just an example file I use, but in this instance its a file 91.99mb
on average 20 seconds
(I have fiber connection, nd the vpn is fast located in same country so is the ftp server).

pklaas
Posts: 9
Joined: Sat Jun 07, 2014 10:47 pm

Re: Bug report: FTP Upload

Post by pklaas » Tue Jun 10, 2014 1:50 am

hi dnobori,
I had the change to test smaller files as well, as I hadto upload a moulefor joomla via ftp.
That didnt go well, none of the times was it complete, eahc time missing some kb's.
I thought I let you know.
Cheers,
Klaas

dnobori
Posts: 228
Joined: Tue Mar 05, 2013 10:04 am

Re: Bug report: FTP Upload

Post by dnobori » Tue Jun 10, 2014 4:05 am

Could you tell me your particular environment?

- Country of the VPN Server
- Country of the target FTP server
- Country of the VPN Client
- The operating system of VPN Server
- Is the VPN Server running on VM? If yes, the platform name or brand name of the VM.

pklaas
Posts: 9
Joined: Sat Jun 07, 2014 10:47 pm

Re: Bug report: FTP Upload

Post by pklaas » Tue Jun 10, 2014 2:41 pm

hi dnobori,
myself, NL,
VPN server + client NL
windows_2008r2_std / Parallels Power Panel

other one is :
Operating System Windows 7 Ultimate 64-bit
Xen HVM)

their other people who use it, mainly iran, (because they have been blocked from many services) they too same exact problem, but I am the one who took it on myself to test outand hopefully that you can provide a solution.

And the strange thing is that sftp works, but most providers they dont provide this option because of server security, its either normal ftp or TLS ftp.

Warm Regards,
Klaas

thisjun
Posts: 2458
Joined: Mon Feb 24, 2014 11:03 am

Re: Bug report: FTP Upload

Post by thisjun » Thu May 21, 2015 5:10 am

Did you use SecureNAT?

denis.kostyanets
Posts: 3
Joined: Tue Feb 27, 2018 8:00 am

Re: Bug report: FTP Upload

Post by denis.kostyanets » Tue Feb 27, 2018 8:03 am

Hi! I use 4.24 build 9651 and still get this error, may be someone know how it fix?

thisjun
Posts: 2458
Joined: Mon Feb 24, 2014 11:03 am

Re: Bug report: FTP Upload

Post by thisjun » Wed Mar 07, 2018 8:10 am

What error did you see?

denis.kostyanets
Posts: 3
Joined: Tue Feb 27, 2018 8:00 am

Re: Bug report: FTP Upload

Post by denis.kostyanets » Mon Mar 19, 2018 9:39 am

Hi!
I have not see an error. Just when I use connection to ftp through vpn, files uploaded are not full, just have missing couple KB from each file.
Also connection to http server that used for upload files have some troubles when you try to upload files. Any error on servers I do not find.

thisjun
Posts: 2458
Joined: Mon Feb 24, 2014 11:03 am

Re: Bug report: FTP Upload

Post by thisjun » Thu Mar 29, 2018 5:30 am

Do you use SecureNAT?

What OS do you use for the server?

denis.kostyanets
Posts: 3
Joined: Tue Feb 27, 2018 8:00 am

Re: Bug report: FTP Upload

Post by denis.kostyanets » Tue Apr 10, 2018 8:19 am

Yes, I use SecureNAT
For server I use Windows server 2016.

thisjun
Posts: 2458
Joined: Mon Feb 24, 2014 11:03 am

Re: Bug report: FTP Upload

Post by thisjun » Fri Apr 27, 2018 3:15 am

Some FTP servers are not compatible with SecureNAT.
Please try localbridge instead of SecureNAT.

Post Reply