Configuration guidance for setting up a virtual hub hosting service

Post your questions about SoftEther VPN software here. Please answer questions if you can afford.
Post Reply
mzi
Posts: 10
Joined: Thu Dec 13, 2018 6:25 am

Configuration guidance for setting up a virtual hub hosting service

Post by mzi » Thu Dec 13, 2018 9:10 am

Hi all

I'm about to setup a new SoftEther VPN Server cluster to host multiple virtual hubs for different customers in a MSP environment.

Our requirements are as follows:
- The setup should be as simple, as robust and as scalable as possible
- Clients from one customer are not permitted to communicate to other clients from the same customer as well as they are not permitted to communicate to other clients from another customer
- The client traffic must pass our datacenter firewall to connect to the server resources of the respective customer (we have separate VLANs/Subnets per customer in a dedicated security zone per customer) as well as surfing the web

My current design considerations are as follows:

- Two virtual machines with Debian Linux for failover reasons (one cluster controller and one cluster member, additional members could be added as the load increases)
- Each server has to NICs, one in a DMZ for the internet facing traffic (client connections) and one in a transfer VLAN for the VPN client traffic
- We create one virtual hub per customer and set the privacy filter mode on each hub
- We assign the IP addresses of the clients statically in a separate client VPN subnet per customer

So far, so good. Now comes my question: To segregate the client traffic from the different customers as good as possible, I see following options:

1. Using e.g. a /16 subnet on the transfer VLAN and the firewall as the default gateway for the transfer VLAN gets the 0.1. Then we use e.g. /24 subnets to assign to the VPN client per customer (starting at 1.1 for the first, 2.1 for the second and so on) to make the firewalling per customer easier (of course we would have to set the subnet on the VPN client to /16 that they can send traffic to the 0.1 as their default gateway). Further, we may have to use ACLs on the virtual hub to block traffic between the hubs (or is this already handled by the privacy filter mode???) and set other filters, e.g. for broadcast etc. to reduce the number of packets in the big /16 Segment
2. Using a /24 as transfer VLAN to the firewall, create one virtual router, connect all hubs to this virtual router and assign different /24 for the clients which will be routed over the central virtual router and from there to the firewall. Further using ACLs to prevent traffic between the clients of different hubs, as the router knows every /24 of the hubs and would route traffic to the other hubs by default i guess?

Which scenario would be the best, 1. or 2.? Any other ideas or considerations?

Any feedback would be highly appreciated.

Thanks in advance and kind regards,

Marco

davidebeatrici
Posts: 33
Joined: Tue Aug 28, 2018 6:44 am

Re: Configuration guidance for setting up a virtual hub hosting service

Post by davidebeatrici » Thu Dec 13, 2018 11:16 pm

Hi,

Privacy filter mode blocks packets between sessions with the setting enabled, independently of the user's virtual hub.

Scenario #1 is probably simpler and more efficient, considering that only a virtual router is needed.

Best regards,
Davide

mzi
Posts: 10
Joined: Thu Dec 13, 2018 6:25 am

Re: Configuration guidance for setting up a virtual hub hosting service

Post by mzi » Fri Dec 14, 2018 12:11 am

Hi Davide

Thank you for your feedback, very appreciate it! Good to know, that privacy filter mode blocks packets between all session with the settings enabled, regardless on which hub they connect.

Concerning Scenario #1: I would rater use bridging to the transfer VLAN than routing if possible, but I have no cluster experience so far, so I don't know if this is the right approach. As far as I understood the documentation correctly in the meantime, if i would use a virtual router, I have to configure it on each cluster member manually and therefore configure it with independent IP addresses on each cluster member to prevent conflicts I guess? But how would than the routing work with different IPs? I guess I'm missing something here. Any advice?

Thanks and kind regards,

Marco

davidebeatrici
Posts: 33
Joined: Tue Aug 28, 2018 6:44 am

Re: Configuration guidance for setting up a virtual hub hosting service

Post by davidebeatrici » Fri Dec 14, 2018 1:13 am

Hi Marco,

The virtual L3 switch function is not designed to be used with the clustering function.

You should use SoftEther VPN to host the isolated L2 virtual hubs in conjunction with an external L3 router which could be iptables on Linux, for example.

One TAP interface per virtual hub would be ideal for a simple end efficient configuration.

Best regards,
Davide

mzi
Posts: 10
Joined: Thu Dec 13, 2018 6:25 am

Re: Configuration guidance for setting up a virtual hub hosting service

Post by mzi » Fri Dec 14, 2018 11:48 am

Hi Davide

Thank you for the hint, good to know. But as far as I get the documentation right, it should be possible to bridge all virtual hubs directly to the transfer L2 segment and use en external router (like our datacenter firewall) to route the traffic from all virtual hubs, as long as the subnet mask of all VPN clients machtches. In this case, the VPN clients are all directly bridget to the same L2 network (regardless off on witch cluster member they are connected, as both are bridged to the same L2 network), right? In this way one router would be sufficient. Any downsides of this setup?

Regards, Marco

davidebeatrici
Posts: 33
Joined: Tue Aug 28, 2018 6:44 am

Re: Configuration guidance for setting up a virtual hub hosting service

Post by davidebeatrici » Sat Dec 15, 2018 4:30 am

Hi Marco,

That setup is definitely possible and there shouldn't be any downsides.

Best regards,
Davide

mzi
Posts: 10
Joined: Thu Dec 13, 2018 6:25 am

Re: Configuration guidance for setting up a virtual hub hosting service

Post by mzi » Mon Dec 17, 2018 6:12 am

OK, good to know. Thank you Davide for your support!

Regards, Marco

davidebeatrici
Posts: 33
Joined: Tue Aug 28, 2018 6:44 am

Re: Configuration guidance for setting up a virtual hub hosting service

Post by davidebeatrici » Mon Dec 17, 2018 6:57 pm

You're welcome!

Feel free to ask for support in case you encounter any problems during the setup.

Best regards,
Davide

mzi
Posts: 10
Joined: Thu Dec 13, 2018 6:25 am

Re: Configuration guidance for setting up a virtual hub hosting service

Post by mzi » Sat Dec 22, 2018 6:58 am

Hi Davide

In the meantime I've setup the cluster and everything seems to work as expected, except of the following two things:
- If I set the virtual hub security option "Deny Non-ARP/ Non-DHCP... broadcasts" on a group to limit broadcasts on the network, any network communication (even non broadcast) is not possible on the hub anymore (TCP/ICMP stops working completely). Is this a known bug or is this option incompatible with the "Privacy Filter Mode" option?
- Is it really true that if the cluster controller is unavailable (e.g. during OS or VPN sever patching/rebooting or if it fails) 1. only currently active VPN sessions on a member server are continue to work, 2. currently active VPN sessions on the cluster controller (as it is not running in "cluster controller only mode" at the moment) are terminated and not moved to or reinitiated on a member server and 3. no new VPN sessions can be initiated? At least this are our test results. If this is really true, than this is not a proper high availability or fault tolerant cluster in my opinion, because it has a single point of failure (cluster controller) and I don't understand why it is written that "it is possible to set up a large scale Virtual Hub hosting service that runs 24 hours a day, 365 days a year with no downtime" in the documentation (10.9.2). If this is true, is there a best practice to circumvent this design flaw?

Thanks in advance for your reply.

Kind regards, Marco

davidebeatrici
Posts: 33
Joined: Tue Aug 28, 2018 6:44 am

Re: Configuration guidance for setting up a virtual hub hosting service

Post by davidebeatrici » Sat Dec 22, 2018 6:43 pm

Hi Marco,
If I set the virtual hub security option "Deny Non-ARP/ Non-DHCP... broadcasts" on a group to limit broadcasts on the network, any network communication (even non broadcast) is not possible on the hub anymore (TCP/ICMP stops working completely). Is this a known bug or is this option incompatible with the "Privacy Filter Mode" option?
The "Deny Non-ARP/ Non-DHCP... broadcasts" option is supposed to block only non-ARP/non-DHCP broadcast packets, independently of the "Privacy Filter Mode" option.

We're not aware of any bugs related to the option.
Is it really true that if the cluster controller is unavailable (e.g. during OS or VPN sever patching/rebooting or if it fails) 1. only currently active VPN sessions on a member server are continue to work, 2. currently active VPN sessions on the cluster controller (as it is not running in "cluster controller only mode" at the moment) are terminated and not moved to or reinitiated on a member server and 3. no new VPN sessions can be initiated? At least this are our test results. If this is really true, than this is not a proper high availability or fault tolerant cluster in my opinion, because it has a single point of failure (cluster controller) and I don't understand why it is written that "it is possible to set up a large scale Virtual Hub hosting service that runs 24 hours a day, 365 days a year with no downtime" in the documentation (10.9.2). If this is true, is there a best practice to circumvent this design flaw?
The mentioned limitation and the workaround for it is described in https://www.softether.org/4-docs/1-manu ... Redundancy. The clustering function is designed to realize a stable system with many unstable cluster members. Only the cluster controller is required to be stable.

Best regards,
Davide

mzi
Posts: 10
Joined: Thu Dec 13, 2018 6:25 am

Re: Configuration guidance for setting up a virtual hub hosting service

Post by mzi » Sat Dec 22, 2018 10:04 pm

Hello Davide

Thank you for your prompt response.
The "Deny Non-ARP/ Non-DHCP... broadcasts" option is supposed to block only non-ARP/non-DHCP broadcast packets, independently of the "Privacy Filter Mode" option.

We're not aware of any bugs related to the option.
OK, good to know that this options should not interfere and the option itself should work as expected. Strange that all traffic stops if we enable it... I will have to do some more testing as soon as I have more time, maybe I will find out why it was not working and get back to you if I find out more.
The mentioned limitation and the workaround for it is described in https://www.softether.org/4-docs/1-manu ... Redundancy. The clustering function is designed to realize a stable system with many unstable cluster members. Only the cluster controller is required to be stable.
OK, good to know that it is a known limitation and I have not just misunderstood the concept or overlooked some configuration options. And thank you for the hint to the documentation. As our SE servers are running on virtual machines and are backed up automatically, we already have a good/quick recovery option if the cluster controller fails, but we are looking for a way to do maintenance works on the servers (OS / SE patching) without impact for our users.

Here is what I think about: We plan to build a second independent cluster in our secondary datacenter anyway (e.g. for site recovery). As we are using Ansible to deploy/configure virtual Hubs and Users (mainly by invoking vpmcmd of course) we plan to configure both clusters 100% identically and deploy a second connection / adapter on all clients with higher metric to connect to the second site in parallel. If the primary cluster is unavailable due to maintenance works, the traffic should automatically be routed over the second site. I will test this scenario as soon as we have setup the secondary cluster.

Kind regards, Marco

davidebeatrici
Posts: 33
Joined: Tue Aug 28, 2018 6:44 am

Re: Configuration guidance for setting up a virtual hub hosting service

Post by davidebeatrici » Sun Dec 23, 2018 4:20 pm

Hello Marco,
OK, good to know that this options should not interfere and the option itself should work as expected. Strange that all traffic stops if we enable it... I will have to do some more testing as soon as I have more time, maybe I will find out why it was not working and get back to you if I find out more.
Excellent, thank you very much.
OK, good to know that it is a known limitation and I have not just misunderstood the concept or overlooked some configuration options. And thank you for the hint to the documentation. As our SE servers are running on virtual machines and are backed up automatically, we already have a good/quick recovery option if the cluster controller fails, but we are looking for a way to do maintenance works on the servers (OS / SE patching) without impact for our users.

Here is what I think about: We plan to build a second independent cluster in our secondary datacenter anyway (e.g. for site recovery). As we are using Ansible to deploy/configure virtual Hubs and Users (mainly by invoking vpmcmd of course) we plan to configure both clusters 100% identically and deploy a second connection / adapter on all clients with higher metric to connect to the second site in parallel. If the primary cluster is unavailable due to maintenance works, the traffic should automatically be routed over the second site. I will test this scenario as soon as we have setup the secondary cluster.
A replicated cluster would definitely be the best option for redundancy. That way your infrastructure should be entirely covered in case something happens to the primary cluster.

Best regards,
Davide

mzi
Posts: 10
Joined: Thu Dec 13, 2018 6:25 am

Re: Configuration guidance for setting up a virtual hub hosting service

Post by mzi » Mon Dec 24, 2018 6:53 am

Thank you Davide for your feedback and support, very appreciate it! I’ll keep you posted as soon as I had time to do some testing on the broadcast issue.

Short cross post question: Have you seen my other post concerning binding/tieing a physical machine to a specific VPN user to prevent moving a VPN client configuration to a non-corporate-managed machine (viewtopic.php?f=7&t=63693)? I guess that there is currently no option to meet this requirement as nobody responds?

Have some nice days off in the following two weeks!

Regards, Marco

davidebeatrici
Posts: 33
Joined: Tue Aug 28, 2018 6:44 am

Re: Configuration guidance for setting up a virtual hub hosting service

Post by davidebeatrici » Mon Dec 24, 2018 7:51 pm

You're welcome.

I actually didn't see your other post, thank you for linking it to me. I replied.

I wish you a Merry Christmas!

Best regards,
Davide

Post Reply