[guide] pfsense VM on QNAP in 2020

Introduce yourself to us and other members here, or share your own product reviews, suggestions, and tips and tricks of using QNAP products.
Post Reply
User avatar
Moogle Stiltzkin
Ask me anything
Posts: 9079
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: [guide] pfsense VM on QNAP in 2020

Post by Moogle Stiltzkin » Tue Jun 09, 2020 1:05 pm

jaysona wrote:
Tue Jun 09, 2020 12:15 pm

That's the point of marketing, it's called make as much money as possible by stretching facts as much as possible. 2018 was a long time ago in QNAP land and VirtualStation. A lot has been learned about the glue (VirtualStation) used to integrate KVM/QEMU in QTS since then. ;)

CVE entries are only made after public disclosure, CVE public disclosure is the tip of the iceberg. By the time a CVE number is assigned, the horses have long left the barn.

If you start to scratch and dig and the reported QNAP vulnerabilities you will see that the exploits go after the QNAP integration code and not the code of the program/apps that QNAP is integrating.

For example, QNAP writes the cgi code that gets exploited, but the httpd server that suffers the attack due to the poor cgi code is only indirectly exploited.

you mean stuff like this?

Remote Exploit ShellShock Vulnerability CVE-2014-6271: 2 Easy Methods
https://www.youtube.com/watch?v=sY-A38zi_GU
Shellshock attacks target QNAP's NAS boxes, FireEye says

The security vendor said the attacks are some of the first seen using Shellshock targeting embedded Linux, which QNAP’s devices run, James T. Bennett and J. Gomez of FireEye wrote in a blog post on Wednesday.

“These attacks result in the hackers having a root level remote shell, gaining full access to the contents of the NAS,” they wrote.

Shellshock is a two-decades-old flaw in Bash, a command-line shell processor present in most Unix and Linux systems. Its discovery last week has set off a scramble to assess the potential risk, as it is easy to exploit and gives attackers full control over a vulnerable server.


QNAP warned on Sunday that its Turbo NAS products were vulnerable, advising administrators to disable Web administration, Web server, WebDAV and other applications and services that use a Web-based interface.

FireEye said taking that precaution may not mitigate the threat, as attackers could find another vulnerable entry point into an organization’s systems and laterally move in order to find a NAS device.

The attack tries to get NAS devices to download a script from a remote server. That script then uploads an SSH (secure shell) key to the local authorized_keys file, which allows the hackers to log in without a password in the future, FireEye wrote.

Also installed is an ELF executable, which is a Linux backdoor that provides shell access. FireEye said that file is named “term_x86_64” or “term_i686.” The servers hosting the malware are in the U.S. and Korea.

In some instances, the backdoor listens for a connection on port 58273. The backdoor serves up a shell if a packet is sent with the text “IAMYOURGOD,” FireEye wrote.
https://www.pcworld.com/article/2690932 ... -says.html

The attack targets a QNAP CGI script, /cgi-bin/authLogin.cgi, a well known vector for Shellshock on QNAP devices," Johannes B. Ullrich, head of the Internet Storm Center at the SANS Institute, wrote in the blog post published Sunday. "This script is called during login, and reachable without authentication. The exploit is then used to launch a simple shell script that will download and execute a number of additional pieces of malware."
https://thehackernews.com/2014/12/malwa ... -hack.html


pfSense and the CVE-2014-6271 ("shellshock") bash exploit.
September 25, 2014
By Jim Thompson

tl;dr: If you’re having shell problems I feel bad for you son. I got 99 problems but bash ain’t one.

If you’ve not heard, Stephane Chazelas discovered a vulnerability in bash, related to how environment variables are processed: trailing code in function definitions was executed, independent of the variable name. In many common configurations, this vulnerability is exploitable over the network.

NIST has assigned a CVSS of 10 CVE-2014-6271. POCs are starting to appear.

So the question becomes, “is pfSense® affected?”

The short answer is: Unlikely, though there are three packages which could lead to an exploit. The base system of pfSense does not include bash. Since bash isn’t on the system, the problem is reduced to packages.

Three packages are affected, and only one is commonly used. The affected packages are:

Anyterm – This package contains bash in its binaries which are in the git repo, not a .pbi or .tgz. This package will simply be retired as it is unmaintained and rarely used. We will review all packages, and any which contain binaries which we have not built from source will be removed or re-engineered such that we can compile from source.

Freeswitch-dev – Runs pkg_add for bash. This package is not actively maintained, and can likely be safely removed from the list of packages with minimal community impact.

FreeRADIUS2 – Adds bash via pkg_add using FreeBSD’s 8.3-RELEASE package set if the user activates Mobile-One-Time-Password (varsettingsmotpenable). We’re looking into the best way to fix it.

Mailscanner – Includes bash also, will be fixed shortly.

Given the lack of impact to pfSense software version 2.1.5 or the pfSense 2.2-BETA images, no fix is required, so we don’t plan any release in response to this issue.

UPDATE: Affected packages have been updated or removed. Full details are in the security announcement which was posted this afternoon. -jimp
https://www.netgate.com/blog/pfsense-an ... ploit.html
gonzopancho

FreeBSD != Linux, friends.

The base system doesn't include bash, so unless it's being pulled in another way we can't see, pfsense is not affected.

Unless you've loaded one of three packages, there is no bash binary on the system.


The affected packages are:

Anyterm Contains bash in its binaries which are in the git repo(!), not a .pbi or .tgz. We're removing the package entirely from the repo. No archive. It's not worth keeping. Bye bye. I've been ** internally about packages we didn't compile. Now everyone understands why.

Freeswitch-dev Runs pkg_add for bash. Unmaintained package. Could probably be safely removed.

FreeRADIUS2 Adds bash via pkg_add using FreeBSD's 8.3-RELEASE package set if the user activates Mobile-One-Time-Password (varsettingsmotpenable). Commonly used package, though we are unsure if the maintainer is still around. Will be deactivated for 2.0.x but kept for 2.1+. For 2.1 we can either build/host an up-to-date tgz for it to pkg_add to minimize changes to the code in the package or build bash into the .pbi and adjust its paths/code to handle that better. We favor adding it to the PBI so that if it happens in the future we need only build a new PBI as usual.

Overall, not a huge impact.

EDIT: The Mailscanner package includes bash also, and will be fixed shortly.
Mateh-
bolapara-

This really isn't a BSD vs. Linux thing. Any system with bash installed is vulnerable. There are Linux systems without bash installed by default too.
Yeah but what he is communicating here is that it doesn't come by default on FreeBSD.
https://www.reddit.com/r/PFSENSE/commen ... hellshock/

A Vulnerability in CGI Based Web Products Could Allow For Unauthorized Redirection of Traffic
07/21/2016

OVERVIEW:
A vulnerability has been discovered in a wide variety of Common Gateway Interface (CGI) based web products, which could allow for unauthorized redirection of traffic. This vulnerability exists due to a flaw in the use of the HTTP Proxy environment variable. This vulnerability can be exploited to perform remote man in the middle attacks, cause Denial of Service (DoS) conditions on the affected server, or leverage the affected server to perform Distributed Denial of Service (DDoS) attacks on a third party target.

SYSTEMS AFFECTED:
Many applications and systems with a reliance on CGI or CGI-like environments are affected Apache|Drupal|Go|IIS|NGINX|PHP|Python

TECHNICAL SUMMARY:
A vulnerability has been discovered in a wide variety of CGI-based web products, which could allow for unauthorized redirection of traffic. This vulnerability exists due to a flaw in the use of the HTTP Proxy environment variable.This vulnerability can be exploited when application code is running on CGI or a CGI-like server. HTTP request headers are merged into a specific variable under keys beginning with HTTP. This information is what getenv reads from. When a user submits a request that contains a Proxy header, the header appears to the application as getenv('HTTP_PROXY'). Some common application libraries are trusting this value, even when run in a CGI/SAPI environment.

This vulnerability can be exploited to perform remote man in the middle attacks, cause Denial of Service (DoS) conditions on the affected server, or leverage the affected server to perform Distributed Denial of Service (DDoS) attacks on a third party target.

RECOMMENDATIONS:
We recommend the following actions be taken:

· Block the Proxy header for applications running PHP or CGI.

· Apply appropriate patches provided by vendors immediately after appropriate testing.

· Verify no unauthorized system modifications have occurred on system before applying the patch.

· Run all software as a non-privileged user (one without administrative privileges) to diminish the effects of a successful attack.

· Apply the principle of Least Privilege to all systems and services.
https://www.cisecurity.org/advisory/a-v ... f-traffic/


https://www.securezoo.com/2020/05/450k- ... abilities/

https://portswigger.net/daily-swig/shie ... y-released



so you are saying these kinds of attack still works even if you don't expose/port forward your pfsense vm router (via virtual station)? :'
Last edited by Moogle Stiltzkin on Tue Jun 09, 2020 6:48 pm, edited 9 times in total.
NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[Backup] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) single disks.
[^] QNAP TS-659 Pro II
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228
[^] QNAP TS-128
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Win10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin

User avatar
rafale
Know my way around
Posts: 184
Joined: Tue May 12, 2015 1:53 pm

Re: [guide] pfsense VM on QNAP in 2020

Post by rafale » Tue Jun 09, 2020 1:26 pm

I have been running pfSense in my main NAS for a few years now and it definitely has it quirks but it has worked out really well. The main thing to deal with is that the QNAP won't have access to internet when it boots and will pop errors and if you run a firmware upgrade requiring to upgrade virtual station at the same time, you may be in for some fun.
The security risks exist but may be blown out of proportion in my opinion. The QNAP QTS is a unix distro (or if you want the GUI on top of the QNAP OS) and Virtual Station as I found out is just a GUI which launches a modified version of KVM/QEMU. It isn't any different than running QEMU on ubuntu or ESXi or Proxmox. There aren't any more layers of virtualization. You can verify that by looking at the processes running on the QNAP. Getting your QNAP cgi attacked requires getting through the router first if you are outside the LAN which should be as difficult as going through pfSense on bare metal.

I have had thoughts about moving it to another machine but have not found anything appropriate for my use. For power efficiency I was looking for an 8th gen U series CPU passively cooled with at least 2 gigabit NIC for WAN and at least one 2.5gigabit port for LAN. (I run dual WAN). There are many options with a dual core 7th generation Core i5 on aliexpress but nothing with an 8th gen and 2.5 gigabit ports. I proceeded to build my own proxmox server which potentially could run a lot more VMs and can support snapshots and backup of VMs.
Current Config:
TVS 872XT, i9 9900T ES, intel X550-T2, Zotac RTX 2070 mini (with pico PSU as secondary), 64GB DDR4 2666MHz, 2x generic Phison E12 1TB M.2 SSD. 4x WD Red 8TB, 4x WD Purple 4TB

Previous and Sold:
TVS 1282, i7 6700K, intel X550-T2,Mellanox ConnectX3, Zotac RTX 2070 mini, 64GB Crucial DDR4, 2xMicron1100 2TB SATA, 2x Crucial C300 128GB Cache, 2x850 EVO 500GB M.2. 4x WD Red 8TB, 4x WD Purple 4TB

User avatar
Moogle Stiltzkin
Ask me anything
Posts: 9079
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: [guide] pfsense VM on QNAP in 2020

Post by Moogle Stiltzkin » Tue Jun 09, 2020 1:42 pm

rafale wrote:
Tue Jun 09, 2020 1:26 pm
I have been running pfSense in my main NAS for a few years now and it definitely has it quirks but it has worked out really well. The main thing to deal with is that the QNAP won't have access to internet when it boots and will pop errors and if you run a firmware upgrade requiring to upgrade virtual station at the same time, you may be in for some fun.
thx for the feedback. this is exactly what i wanted to know, from an experienced users having used this for far longer than i have :D

is this being used as an edge router? (this is my current usage)


you made a good point especially about updating virtual station. if that qts or virtual station update had issues, then yeah.... :shock: i'll be having to pay more attention to any issues in regards to virtual station. if a release gets botched, there goes my internet :(
https://www.qnap.com/en-us/app_releasen ... hoose=QKVM

rafale wrote:
Tue Jun 09, 2020 1:26 pm
I have had thoughts about moving it to another machine but have not found anything appropriate for my use. For power efficiency I was looking for an 8th gen U series CPU passively cooled with at least 2 gigabit NIC for WAN and at least one 2.5gigabit port for LAN. (I run dual WAN). There are many options with a dual core 7th generation Core i5 on aliexpress but nothing with an 8th gen and 2.5 gigabit ports. I proceeded to build my own proxmox server which potentially could run a lot more VMs and can support snapshots and backup of VMs.
this was my problem as well. headache trying to find what parts to need, and put them all together on an affordable budget. and i didn't want to spend that much and only find out i got the wrong part, or trouble installing the hardware. and the prebuilt systems, either over priced vs others, or there many options, and unclear which can sufficiently operate for vpn + suricata usage (for 100-500 mbps broadband)

at least now i've succeeded trying out pfsense so i think i can manage with pfsense now that i have some experience.

i'll take my time and see what nuc to get eventually :'
NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[Backup] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) single disks.
[^] QNAP TS-659 Pro II
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228
[^] QNAP TS-128
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Win10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin

User avatar
Moogle Stiltzkin
Ask me anything
Posts: 9079
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: [guide] pfsense VM on QNAP in 2020

Post by Moogle Stiltzkin » Tue Jun 09, 2020 2:11 pm

i have a qnap helpdesk remote session active.

i only can look at qts active sessions , and the helpdesk app to know qnap is accessing remotely.

but how do i check any remote accessed ips logging into my network this way from pfsense? any ideas



*update

got back from helpdesk
Hi Moogle,

After checking, this issue seem to be caused by Qfinder utility, since your NAS virtual switch 1 set "do not assign IP address" this option, therefore, the Qfinder utility cannot get correct IP address from the NAS LAN adapter 1 & show incorrect IP address, could you please help to provide which version of Qfinder utility have you use it and Windows OS version, then I can inform our 2nd line help to analysis this issue further.


Thank you.
i reported an issue where qfinder had 2 ip address. one of them is my qnap nas gateway lan ip, which is fine. But the other one had some odd ip there. was worried i may have misconfigured pfsense qnap or something. so i asked helpdesk to check that out.


i checked firewall seemed okay, so i reckon it's not a misconfiguration my end, but probably qfinder has a bug? not sure that is why i'm asking helpdesk to check it out


*update

in regards to my question, is pfsense qnap safe for edge router use, this was the reply i got
Hi Moogle,

The pfSense VM OS should protect your NAS device when your NAS set virtual switch 1 with "do not assign IP address" this option & bundle it with pfSense LAN adapter 1 and NAS LAN adapter 1, but we still suggest you update the NAS latest version of firmware & APPs to protect your NAS device.

You can manually change the VM OS type & version when you shutdown the pfSense VM OS in your NAS Virtualization station.

We could not guarantee use previous VM OS snapshot to recovery pfSense VM OS can normal work, but you can try it in your NAS device.

*update

confirmation qfinder bug needs to be fixed sometime. alien ip was not due to any pfsense misconfiguration on my end
Hi Moogle,

After the internal check, our 2nd line has confirm the Qfinder utility get incorrect NAS IP address when virtual switch set "do not assign IP address" this option.

They will inform our development team help to analysis this issue & find the solution, once they have any update, I will update it in this ticket.



Thank you.
Last edited by Moogle Stiltzkin on Wed Jun 10, 2020 3:37 pm, edited 6 times in total.
NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[Backup] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) single disks.
[^] QNAP TS-659 Pro II
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228
[^] QNAP TS-128
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Win10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin

User avatar
Moogle Stiltzkin
Ask me anything
Posts: 9079
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: [guide] pfsense VM on QNAP in 2020

Post by Moogle Stiltzkin » Tue Jun 09, 2020 2:54 pm

I found a guide which was roughly what i was doing for setting up pfsense on the qnap. this was posted as far back as September 5, 2016
https://trainingrevolution.wordpress.co ... ap-ts-251/


this is his network virtual switch topology in qts *note: he is using an older qts so keep that in mind
Image


There were two settings that had a significant impact on performance:

-Hardware Checksum Offloading
-Device Model Selection


During my initial testing I found that my download speed was awful 2.72 Mbps, that is not a typo it was 2.72 Mbps after multiple tests. Checking the box to Disable hardware checksum offload restored my download speed to 255.53 Mbps.

Open a web browser connection to http://192.168.1.1 and logged in as admin with password pfsense. From the top menu select System –> Advanced –> Network and then select the check box to disable Hardware Checksum Offloading.



When I originally installed pfsense I used the Device Model Intel Gigabit Ethernet and although it provided a respectable download speed of 84.74 Mbps, switching to the Device Model Virtual Gigabit Ethernet provided the best download speed at 255.33 Mbps.

In addition to conducting several performance tests I was also curious about CPU utilization. I monitored CPU Utilization during testing and found that utilization rates were relatively high spiking anywhere from 40% up to 60% during most tests.

one thing i noticed in the guide was he selected unix and freebsd. is this important?
Image

because for my setup i put generic for both os type and version (afaik i didn't notice any issues), because i didn't know what to put :' is this going to be a problem? or should i redo the vm again?
Image


*update

When you power off the pfsense vm. then you can change from generic to the freebsd latest version. then power on the pfsense vm again.

it seemingly works, so i don't need to resetup the vm from scratch. i'll however create new snapshots (easy to do) just in case they don't work after this change.





also he says he uses "Device Model Virtual Gigabit Ethernet", but mine is setup to use virtio (which was qnaps setting when setup using vm market). I didn't notice any broadband performance issues using the virtio for networking. I ran a couple of tests earlier.


but for hard drive, i think it should be configured as "writethrough" virtio, for decent performance. I was having issues with the defaults based on vm markets default settings for it (which had no hard drive cache enabled).



other user posts in regards to using pfsense on qnap

https://forum.netgate.com/topic/131972/ ... n-qnap-nas
https://www.reddit.com/r/qnap/comments/ ... h_pfsense/
https://www.reddit.com/r/PFSENSE/commen ... _products/

[–]mauvehead
14 points 2 years ago

So your NAS is your router? That's seems like a bad idea.

[–]pfsense-ivork
[S] 2 points 2 years ago

Depends how you look at it. NAS is a server and in this case it's a server capable of virtualization. pfSense will be a VM running on top of it. Most of /r/homelab runs pfSense virtualized on top of their servers, along with other VM's which often include NAS VM's.

[–]kingrpriddick

I trust it (pfSense) more than consumer routers with their backdoors and otherwise sloppy software. A hacker would need remote execution on the pfSense vm before even trying to break out of the VM, and that's why I think this isn't too bad.

Also, alot of people already use their consumer routers as file servers that's why SmallNetBuilder still does a USB speed test

[–]gollito

For home users or small businesses that use crappy off the shelf products it is a way better solution. Not knowing how qnap does virtualization I can't say for certain but if done properly I'd say it is pretty low risk
[–]pfsense-ivork

QNAP Virtualization Station is based on KVM.
richtemark Sep 9, 2018,

Hello,

I run pfSense in a QNAP TS-251+. I upgraded the QNAP to 16GB RAM; in the virtualization station, I have allocated 2GB for the virtualization station itself, 4 GB of RAM as well as 2 out of the 4 cores for the pfSense VM. I did not use the image provided on the QNAP page but the "original" pfSense ISO for installing the VM.

The whole setup works well after some tweaking and experimenting. The QNAP has two physical network adapters. One is connected to my cable internet, the other one to my router (that serves as WLAN repeater only). Both physical network adapters have been assigned a virtual network adapter in QNAP.

I use pfSense as router for establishing my internet connection, DHCP server for the devices in my network, snort and connecting as client to three different VPN servers at the same time.

Since I am living in China, I cannot really say reliable things about the connection speed but the whole setup works well. All my network devices connected via wifi have a VPN connection to the free world; my china devices in the network are blocked via firewall rules from establishing any connections to the outside world.


potential issue?
fabrizior Jan 4, 2019,

Ran into an interesting issue with the QNAP's default gateway config after switching my router from an external device to virtualized PFSense on the QNAP.

Wondering if anyone else has noticed or run-into this?


My TVS-1282T will no longer acknowledge it's default gateway route now that my "router" has an IP address on the QNAP's own Virtual Switch. There seems to be some additional configuration needed or perhaps a bug with the QNAP OS/config utilities in this regard?

Installed PFSense in a VM on my QNAP NAS using the network topology itemized below. Everything seemed fine until I discovered that the QNAP refused to assign its own default gateway to an IP allocated within a virtual switch config assigned to a VM (for the internal LAN interface to pfSense).

pfSense seems to be operating just fine, everything else on my network can access the configured "default gateway" to the PFSense LAN interface - the only problem came on the QNAP side of things where it is refusing to talk to it's gateway IP anymore, and wouldn't let me confirm/update the setting to the 192.168.4.1 IP since it belonged to a virtual device.

The virtual switch
Any suggestions on how to maybe get this working without having to move pfSense to a separate HW device?

Physical Topology:
wan: cable modem -> QNAP eth4
lan: switch LAG (ports 21-23) -> QNAP bond0 (eth1+eth2+eth3)

QNAP switch config:
virtual switch 1: qnap bond0, static IP on 192.168.4.0/24, gw: 192.168.4.1
virtual switch 4: qnap eth4, no IP config, no dhcp... (assigned dhcp by modem in pfSense)

VM network config for pfSense:
Adapter 1 - vmeth0 (virtIO driver) <- QNAP virtual switch 4 (WAN), dhcp via cable-modem
Adapter 2 - vmeth1 (virtIO driver) <- QNAP virtual switch 1 (LAN), static IP 192.168.4.1

TIA,
-Fabrizio
Eecam Jan 2, 2020,
@fabrizior , I'm running into the same problem. Did you ever got this figured out?

Ffabrizior Jan 2, 2020,
@ecam nope... with all the other containers, VM, and storage dependencies I have on my qnap array, I decided that it wasn’t the best place to be running my firewall and intrusion-protection (even as a HA backup ) Then there’s the risks trying to keep the interfaces and routing working and isolated while “fighting” with all the brain-dead err.. questionable update/config decisions that qnap makes during FW updates such...

Decided to dedicate some additional HW instead:
ProtectCLI FW6 Vault Model: FW6C
i suspect his issue is that the virtual switch in virtual switches was misconfigured.

Is he saying that he cannot connect to his QTS gateway for his NAS? after doing his setup. Only the pfsense works, but not his qts gateway?

For me i resolved that by following what rlsted suggested for his virtual switch guide. I think this is where Fabrizio ran into a problem :'

This is my virtual switch setup
rsted-

Short setup instructions cause someone asked. Please note that some networking/pfsene knowledge is required to set this up.



Install pfsense using this guide.

https://www.qnap.com/en/how-to/tutorial ... -qnap-nas/

I used the pfsense ovf provided by QNAP. The basic setup in the tutorial should be enough to get you going. (one WAN side NIC - one LAN side NIC)

Setup your network on the "network and virtual switch" Application on the QNAP according to this picture.

Image

Do not use any NAT or DHCP features from QNAP - all is done by pfsense. When configuring the QNAP vSwitches don't set an IP adress on the pfsense WAN side virtual switch since pfsense WAN NIC will pull an IP from Modem/Gateway DHCP. The pfsense LAN side should be configured as static IP inside the pfsense LAN network range. The QNAP Webinterface should be accessable through this IP.

Connect your PC to the QNAP NIC Port 2 - you should also get an IP address from pfsense DHCP Server.

Set default gateway on QNAP to the Pfsense Internal Network Switch ((see picture).

all done



Additional hints:

* it's tricky to set this up without locking yourself temporarily out. Some networking knowledge is useful if that happens.

* KVM Switches behave a little slower then physical switches. Take your time and hit ipconfig /renew till you get an IP.

* don't forget to set pfsense VM to start when the QNAP starts. otherwise you won't get an IP adress when the QNAP reboots.
https://www.reddit.com/r/qnap/comments/ ... erface_as/


if what Fabrizio is saying he is using a static set in virtual switch lan, but wants to change it to something else, then i think he needs to
1. go virtual station > shut down the pfsense vm. (if he can't access qts, then skip to the soft resetting part)
2. pull out the wan ethernet cable from the pfsense nas router (qnap nas)
3. go to virtual switches, change the lan virtual switch static lan ip to the new one (it should be a lan ip not in the DHCP range in pfsense). apply.
4. power on the pfsense vm in virtual station.
5. on desktop pc, go command line and type (maybe is not required, can skip to step 6. or if that doesn't work, try step 5 first. or if that doesn't work, try unplug the lan ethernet for the qnap nas, then replug back in)

Code: Select all

ipconfig/release
then

Code: Select all

ipconfig/renew
https://helpdeskgeek.com/networking/rel ... p-address/

6. then use qfinder, refresh see if your qnap gateway is now using the new lan ip
1. remove wan cable from qnap nas
2. if locked out of qnap nas gateway, then have to do a soft reset (1 beep)
https://www.qnap.com/en/how-to/knowledg ... explained/

3. qfinder check for gateway. now login to that. then change password. shutdown pfsense vm. then re-configure virtual switches (refer to rlsted's guide). when done, power up pfsense vm. go qfinder refresh see if it works?

some clarification.


Pfsense

Interfaces > LAN (vtnet1) = this is where your pfsense static lan ip gateway is edited. e.g. 192.168.1.10


then on the QNAP qts side, there is the lan virtual switch. there are 3 switches to be exacted. the physical switch, which is paired to the virtual lan switch. which in turn is bridged to the pfsense virtual lan switch. But the one concerning the gateway is the "virtual lan switch" which you set a static lan ip (this is how you access your qnap nas gateway e.g 192.168.1.100).


and let's say in pfsense dhcp server, the DHCP range for the lan is 192.168.0.30 - 192.168.0.90


see, the pfsense router gateway is DIFFERENT from the qnap nas gateway. and assigned in the different places. Also note they are both OUTSIDE the ranges of the DHCP (or you will have problems).


Qnap's own advise to configure virtual switches is as follows
Hello,

Thank you for contacting QNAP.

step1: Please delete all virtual switch first,

step2: create a virtual switch 1 to bundle the NAS LAN adapter 1 & pfSense adapter 1(WAN) with basic mode in your NAS network & virtual switch APP > virtual switch page,

step3: then create another virtual switch 2 with advance mode to bundle the NAS LAN adapter 2 & pfSense adapter 2(LAN) > select "do not assign IP address" this option, you can reference our tutorial as below link to setup pfSense setting, the pfSense will get IP address from NAS LAN port 1 & provide DHCP IP address to NAS LAN port 2.

https://www.qnap.com/en/how-to/tutorial ... a-qnap-nas

https://www.qnap.com/en/how-to/tutorial ... ual-switch

you decide which is the correct method. i just copied rlsted since his works for me :)
Last edited by Moogle Stiltzkin on Wed Jun 10, 2020 12:04 pm, edited 2 times in total.
NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[Backup] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) single disks.
[^] QNAP TS-659 Pro II
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228
[^] QNAP TS-128
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Win10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin

User avatar
jaysona
Easy as a breeze
Posts: 279
Joined: Tue Dec 02, 2008 11:26 am
Location: Somewhere in the Great White North

Re: [guide] pfsense VM on QNAP in 2020

Post by jaysona » Tue Jun 09, 2020 10:16 pm

Moogle Stiltzkin wrote:
Tue Jun 09, 2020 1:05 pm

you mean stuff like this?

Remote Exploit ShellShock Vulnerability CVE-2014-6271: 2 Easy Methods
https://www.youtube.com/watch?v=sY-A38zi_GU
Yes, along those lines. The target of shellshock is bash, the vector to access the target is QNAPs poor cgi coding. There is no direct access to bash, but poor cgi code means anyone could access bash.
so you are saying these kinds of attack still works even if you don't expose/port forward your pfsense vm router (via virtual station)? :'
Think about all the layers for a moment.

1. physical network stack - basterdized QNAP OS layer
2. virtual network stack - QNAP glue layers.
3. VirturalStation - more QNAP glue layers.
3. psFsene network - whatever the QNAP glue sends it.

I doubt there is any issue with pfSense, the issue is with all the QNAP glue that pfSense needs to rely on for its protection.

Security devices should have one purpose and one purpose only - security, nothing else.

The NUC device you referenced to earlier is more than adequate. I used to run pfSense on a QOTOM box I picked up on sale for about C$280 on Newegg, and it worked great, especially for full speed VPN connections - of which I had 6 simultaneous tunnels running.

I have since ditched the pfSense device, Asuswrt-Merlin and iptables is more than fine for my gigabit connection and I no longer require wireline VPN speeds, so the AC-86U has more than enough CPU power for primary 940/940 connection needs and I use an AC-68U on my 500/50 backup connection.
H/W: TS-219 Pro / TS-269 Pro / TS-253 Pro (8Gig)
H/W: TS-509 Pro x2 / TS-569 Pro / TS-670 Pro (i7-3770S 16Gig) x2 / TS-853 Pro (8Gig)
O/S: Slackware 14.2 / MS Windows 7-64 (x5)
Router1: Asus RT-AC86U - Asuswrt-Merlin - 384.18
Router2: Asus RT-AC68U - DD-WRT v3.0-r39960M kongac
Router3: Linksys WRT1900AC - DD-WRT v3.0-r43028 std
Router4: Asus RT-AC66U - FreshTomato v2020.3
Misc: Popcorn Hour A-110/WN-100, Pinnacle Show Center 250HD, Roku SoundBridge Radio (all retired)

User avatar
rafale
Know my way around
Posts: 184
Joined: Tue May 12, 2015 1:53 pm

Re: [guide] pfsense VM on QNAP in 2020

Post by rafale » Tue Jun 09, 2020 10:50 pm

Moogle Stiltzkin wrote:
Tue Jun 09, 2020 1:42 pm
is this being used as an edge router? (this is my current usage)
Yes. So all the security concerns apply if someone breaks into the LAN as it exposes the VM to all the QNAP security weaknesses. It makes securing your LAN all the more important.
Moogle Stiltzkin wrote:
Tue Jun 09, 2020 1:42 pm
you made a good point especially about updating virtual station. if that qts or virtual station update had issues, then yeah.... :shock: i'll be having to pay more attention to any issues in regards to virtual station. if a release gets botched, there goes my internet :(
https://www.qnap.com/en-us/app_releasen ... hoose=QKVM
The trick is, before every firmware upgrade to download the latest virtual station package from the QNAP site and upgrade it manually after the firmware is upgraded. This will prevent a lot of pain.
Current Config:
TVS 872XT, i9 9900T ES, intel X550-T2, Zotac RTX 2070 mini (with pico PSU as secondary), 64GB DDR4 2666MHz, 2x generic Phison E12 1TB M.2 SSD. 4x WD Red 8TB, 4x WD Purple 4TB

Previous and Sold:
TVS 1282, i7 6700K, intel X550-T2,Mellanox ConnectX3, Zotac RTX 2070 mini, 64GB Crucial DDR4, 2xMicron1100 2TB SATA, 2x Crucial C300 128GB Cache, 2x850 EVO 500GB M.2. 4x WD Red 8TB, 4x WD Purple 4TB

User avatar
Trexx
Ask me anything
Posts: 5285
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

[guide] pfsense VM on QNAP in 2020

Post by Trexx » Tue Jun 09, 2020 11:06 pm

A few suggestions.

To me a guide is really like a recipe:

List of ingredients (Qnap with this hw standard or better, VS version, PF sense, etc)

Concise, detailed and sequenced steps with a FEW pictures for things that are challenging to put into words.

Step 1 do this
Step 2
Step 3 profit :)

Reference LINKS at the end pointing to specific/relevant info.

This feels more like a working blog into your journey trying to deploy pfsense.

Nothing wrong with that, but it is NOT something that I would use for a guide.

It is also all over the place and veered off into the YouTube abyss too much for my tastes.

A little info is good, 5+ YouTube videos becomes noise.




Sent from my iPhone using Tapatalk
Paul

Model: TS-877-1600 FW: 4.4.3.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x 500GB Evo 860
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
GPU: EVGA GTX 1060 6GB
UPS: CP AVR1350

Model:TVS-673 32GB FW: 4.4.3.x Test/Backup Box
Model:TS-228a FW: 4.4.3.x Test/Backup Box
-----------------------------------------------------------------------------------------------------------------------------------------
NAS RAID Rebuild Times | Live QTS Videos | | QNAP NAS Guide | Information needed when you ask for HELP | QNAP Links, Tutorials, etc.
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq

Yippym
Starting out
Posts: 45
Joined: Wed Apr 11, 2018 5:49 am

Re: [guide] pfsense VM on QNAP in 2020

Post by Yippym » Tue Jun 09, 2020 11:39 pm

Phew, not just me that have put PFSense on QNAP, been running got almost a month now. Got Haproxy, Unifi, Netflow currently running, planning not to expose the switch externally. The PFSense itself is the gateway which I trust more than QNAP itself.

You can check out my experience on installing PFSense on the QNAP QGD-1600P here:
pfSense on the QNAP QGD-1600P – Part 3 https://poyu.co.uk/2020/05/12/pfsense-o ... 0p-part-3/

Or go to the forum post viewtopic.php?f=45&t=154885, I believe it the same thing, just use VM Installer to help set the VLAN up for your Virtual Machine.

User avatar
Moogle Stiltzkin
Ask me anything
Posts: 9079
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: [guide] pfsense VM on QNAP in 2020

Post by Moogle Stiltzkin » Wed Jun 10, 2020 9:25 am

Trexx wrote:
Tue Jun 09, 2020 11:06 pm
A few suggestions.

To me a guide is really like a recipe:

List of ingredients (Qnap with this hw standard or better, VS version, PF sense, etc)

Concise, detailed and sequenced steps with a FEW pictures for things that are challenging to put into words.

Step 1 do this
Step 2
Step 3 profit :)

Reference LINKS at the end pointing to specific/relevant info.

This feels more like a working blog into your journey trying to deploy pfsense.

Nothing wrong with that, but it is NOT something that I would use for a guide.

It is also all over the place and veered off into the YouTube abyss too much for my tastes.

A little info is good, 5+ YouTube videos becomes noise.




Sent from my iPhone using Tapatalk
i don't dispute this to most of what you highlighted. like i implied, this is more of a worklog diary for me in my progress for setting up pfsense on qnap.

i did try to add a tldr summary of the gist of getting it up

and as i suggested, parts people don't need to know can just skip over. nobody is expected to read everything :wink: i just left some links or info that could be useful from what i was checking out on this topic.

i saved people at least 80% of the effort having to browse around finding out info left unsaid or is posted as tidbits all over the web from multiple sources (which i linked). if people think browsing 3 pages on forum is too much effort vs doing their own search over the entire web to find out themselves, well :roll: i just help those that want help. others who think it's not useful for them can go do their own time consuming research :mrgreen: i don't force my noise down anyones throats. they are free to block, skip or not read :wink:

look at it from own viewpoint, the reason for the rush sloppy work, is because i got stuff of my own to do as well, and i don't get paid for this neither :( this is mostly a hobby of mine just exploring the techy options for my own usage (in this case i was replacing my ac68u for a pfsense router. and also changing the old router to a wireless ap as well). rather than keep this stuff to myself i opt to share with others. it may or may not help you while i'm at it, i make no guarantees or promises :(

if someone wants to take the time and effort to make a better guide, i welcome that :wink: more competition is good. but because there was a lack of details i needed on subject, or they were located all over the net. so rather than have a single source guide for me to refer to for my own usage, i had to do this research myself and post what i found out here in this thread.

jaysona wrote:
Tue Jun 09, 2020 10:16 pm
..
i was looking into those qotoms because it is cheaper than a netgate official pfsense device by far. . it's suggested to format and install your own pfsense, considering they come from. china :S

i also wasn't sure if these qotom devices were free from any hardware tampering
https://www.bloomberg.com/news/features ... -companies
https://arstechnica.com/information-tec ... le-as-200/


i was recently looking at hp thin clients, but you need to get the parts yourself (intel nic, msata ssd, ram) and even then you may get the wrong one (prices for hp clients went up in 2020)
https://www.youtube.com/watch?v=cRSZ_pDO1SY



i used asus router before. it's easy to setup and quite capable (especially after using rt merlins firmware which uses close to stock and is able to use stock asus features unlike other projects like tomato that make too many changes from stock). my issue with my model was heating, and the necessecity using big loud fans for it :S

another reason i wanted to move to pfsense rather than stick to an asus router, i wanted to use stuff like pfblocker (i hear pihole works similarly to it) to improve my networking, because not all users on network are tech savvy about adblockers. i haven't yet delved into suricata, i don't have enough ram yet to test that out.

rafale wrote:
Tue Jun 09, 2020 10:50 pm
...
i'll set qpkg update to manual from auto, just in case. i check qts often enough so not a problem for me updating when needed. thx for the tip.
Yippym wrote:
Tue Jun 09, 2020 11:39 pm
....
i'll check that review out :'

it has a qunet switch hm. vminstaller? that might be unique for your model, cauz i didn't see that for my ts-877 in appcenter.
NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[Backup] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) single disks.
[^] QNAP TS-659 Pro II
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228
[^] QNAP TS-128
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Win10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin

User avatar
Moogle Stiltzkin
Ask me anything
Posts: 9079
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: [guide] pfsense VM on QNAP in 2020

Post by Moogle Stiltzkin » Wed Jun 10, 2020 12:50 pm

pfSense 2.4.5-RELEASE-p1 Now Available
https://www.reddit.com/r/PFSENSE/commen ... available/

2.4.5-p1 New Features and Changes
https://docs.netgate.com/pfsense/en/lat ... anges.html


from my brief look, there are cve security patches, so definitely worth updating
NEW NXNSATTACK EXPLAINED
In a research paper published today, academics from the Tel Aviv University and The Interdisciplinary Center in Herzliya, Israel, said they found a way to abuse this delegation process for DDoS attacks.

The NXNSAttack technique has different facets and variations, but the basic steps are detailed below:

Image

1) An attacker sends a DNS query to a recursive DNS server. The request is for a domain like "attacker.com," which is managed through an attacker-controlled authoritative DNS server.
2) Since the recursive DNS server is not authorized to resolve this domain, it forwards the operation to the attacker's malicious authoritative DNS server.
3) The malicious DNS server replies to the recursive DNS server with a message that equates to "I'm delegating this DNS resolving operation to this large list of name servers." The list contains thousands of subdomains for a victim website.
4) The recursive DNS server forwards the DNS query to all the subdomains on the list, creating a surge in traffic for the victim's authoritative DNS server.



This large PAF implies that NXNSAttack is one of the most dangerous DDoS attack vectors known to date, having the ability to launch debilitating attacks with only a few devices and automated DNS queries.
https://www.bleepingcomputer.com/news/s ... os-attack/
https://www.zdnet.com/article/nxnsattac ... s-attacks/

Do not update packages before upgrading pfSense! Either remove all packages or do not update packages before running the upgrade.

i'm using this for production so... i'll wait a bit longer and see what others say about this version.

well... i can easily test it because i use snapshots (the benefits of using vm pfsense), but i will only do this during an off hour when nobody is using the internet ideally :wink:
mujimuji

I upgraded 4 pfSense VMs from 2.4.5 to 2.4.5_1. All upgrades were successful, but after the initial reboot various services did not start automatically, including Open-VM-Tools and pfBlockerNG-devel. I had to restart the VMs a second time to get the services back up.
gniting

Updated from 2.4.5 without issues. Actually it took <5 minutes and I don't have a fast connection, which has me a bit puzzled. The upgrade notice said that it will download 10 things totaling ~65MB.

Post upgrade, I do see memory usage increase by 7%-10%
[–]ShuttleMonkey

Lol yeah, when 2.4.5 came out I upgraded immediately and it broke my network. Had to reinstall...

ObscureCulturalMeme

After the update packages finish downloading it could take 10-20 minutes or more until the upgrade process ends.

Can the web GUI be changed to reflect this fact? The "rebooting, back in 90 seconds" message implies that everything's coming back in 90 seconds. Moreso the repeated "ah **, it's not back yet, trying again in 30 seconds" messages.
[–]corknation


I got burned by this in the past so now i upgrade via the console and i can see when the upgrade is actually complete and safe to power down if needed.
DennisMSmith
Netgate[S]

u/ObscureCulturalMeme

Yes our team is aware of that and is evaluating a change. You can track the progress here.
Description

When a user is performing an upgrade using the Web GUI they are presented with this message after the files are copied and the reboot is about to be executed:

Upgrade is complete. Rebooting in 10 seconds.
Success

The Web GUI then tells them:

Rebooting
Page will automatically reload in N seconds.

That is misleading. The upgrade is not complete. There is much work still to be done after the system reboots. Slower systems with lots of packages might take several minutes before the upgrade is actually completed but the Web GUI user is not presented with any information indicating that is the case.

Anecdotally, this results in users experiencing (probably reasonable, based on the information they are given, since it looks like they only need to wait for the device to reboot) impatience and they pull the power while packages are being installed after the reboot.

This will almost certainly result in a "bricked" device and the upgrade process is blamed as the cause.
https://redmine.pfsense.org/issues/10387
mooky1977

I waited for the first maintenance release, and didn't do the 2.4.5 update.

I backuped my configuration, then rebooted, ssh'd into it and updated via the shell:

Code: Select all

pfSense-upgrade
It was pretty painless. Only snort didn't come back as expected, nor did it spin back via a restart service command, so I reinstalled it over itself, and it came back to life as simple as that.

Besides a bit more RAM usage, seems to work just as well as 2.4.4-p3 did, and if you're on the fence, as long as you don't have obscure hardware, I say go for it.

My hardware is an HP 6000 Pro series, Q6600 CPU, onboard NIC is an Intel chip gigabit, secondary card I installed was also an Intel NIC, 4 GB RAM, and an old Intel 240GB SSD (overkill but I had it lying around), so nothing special.

So far, so good, uptime currently 30 minutes :)



update


when i was trying to update pfsense via web UI, i noticed a similar issue
Can't install or Upgrade packages or OSRESOLVED self.PFSENSE

Submitted 1 year ago * by Catsrules


*Edit SOLVED

Thanks everyone for your help, updating from the cli worked. And after the upgrade it appears the web package manage is working again. I was able to install and remove packages.

So this has been a problem for almost a year now it has just not been a high priority for me to fix. I can't update or install anything on my pfSense, nor can I upgrade the base OS. I am still on 2.4.2 (amd64)

When I try to update from the web interface it just sits at

Please wait while the update system initializes

I have left it for hours and it doesn't do anything else.


I have googled around and found this post,

https://forum.netgate.com/topic/119298/ ... nitializes

Basically they said they needed to disabled IPv6, I did that

System -> Advanced -> Networking -> Allow IPv6

Uncheck to block all IPv6 traffic

But it didn't resolve the problem :(

I am not sure what is the best log file to be looking I am looking at the system.log file at the moment.

When I try to upgrade a package, I get this result over and over again on the system.log file.

https://pastebin.com/uzwQBcMN
It looks like it is stuck in an endless loop. These messages did not start until I clicked the update button on on of the packages. (In this instance it was Status_Traffic_Totals.) But I believe this will happen on any package.

Any ideas? Is there better way to diagnose this issue?
https://www.reddit.com/r/PFSENSE/commen ... ges_or_os/
https://forum.netgate.com/topic/119298/ ... nitializes



If i waited longer would it have displayed any progress in the web ui indicating an update is still actively ongoing? i was gonna wait an hour to find out. So i am unsure if it would have worked had i just waited or not.


so now i am trying to update from console (hoping that it at least would be more verbose, indicating that the updating is in progress). i go to virtual station pfsense vm console.
cmacmahon-netgate

Each blog post we publish for a release has a step by step guide on updating pfSense:

https://www.netgate.com/blog/pfsense-2- ... lable.html

Upgrading to pfSense 2.4.4-RELEASE-p2 Updating from an earlier pfSense 2.4.x release to 2.4.4-RELEASE-p2 is possible via the usual methods:

From the GUI:

Navigate to System > Update Set Branch to Latest stable version (2.4.x) Click Confirm to start the upgrade process

From the console or ssh:

Select option 13 OR select option 8 and run pfSense-upgrade


How to update pfsense vm for qnap - by moogle

doing console option 8 run "pfSense-upgrade" without quotes, also that is a capital "S" for Sense (important). worked for me.
notes: earlier during testing and recover using snapshot revert, i reckon it's better to do a reboot from shell option 5 normal reboot. doing so, i notice that wan uptime shows correctly rather than act wonky due to just simply reverting without doing a reboot after.

in summary, when i made a mistake earlier, i revert, then in shell i reboot. only after that, when it's fully booted, i go to the pfsense vm console option 8 and did the pfSense-upgrade
When you update via console, it will at times say it's doing something, and seemingly frozen, but it's actually still running. don't just assume it's frozen and inactive. just wait.

In comparison to my asus ac68u update procedure, we would have a UI that shows a % progression. But in the cmd, at times there is no %, you just have to wait for it to complete.

Usually you will know when it's done when it asks you "do you want to proceed update and reboot? Y / N ?". at that point type y then enter to proceed.


while finding out how to update pfsense via console i also found other commands. i don't know what the difference is or what they do, so don't simply use them if you don't know either. i'll look into that later.

what does -d do? and what is difference between pkg update upgrade and when to use them?

Code: Select all

pfSense-upgrade -d

Code: Select all

pkg-static update -f

Code: Select all

pkg-static upgrade -f
Major thing is that, you need to update pfsense, before you even attempt to update any other packages according to netgate. so update-pfSense first should be safe enough.

For packages, i think i'll update that via the web ui. But for pfsense update, i will rely on the console method.


anyway, you know when updating pfsense fully completes after you do the reboot prompt and it does all it's updating and reaches the console part with all the numbered options. it will also have mention of the new pfsense version number you are currently running.

At this point you can login to pfsense web ui, then see the new pfsense version is indeed running :mrgreen:
Last edited by Moogle Stiltzkin on Wed Jun 10, 2020 5:58 pm, edited 3 times in total.
NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[Backup] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) single disks.
[^] QNAP TS-659 Pro II
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228
[^] QNAP TS-128
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Win10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin

Yippym
Starting out
Posts: 45
Joined: Wed Apr 11, 2018 5:49 am

Re: [guide] pfsense VM on QNAP in 2020

Post by Yippym » Wed Jun 10, 2020 4:44 pm

Moogle Stiltzkin wrote:
Wed Jun 10, 2020 9:25 am
i'll check that review out :'

it has a qunet switch hm. vminstaller? that might be unique for your model, cauz i didn't see that for my ts-877 in appcenter.
Ah, I assume the VM Installer is available for all.

All it does is assigned a VLAN 4001 (WAN) and VLAN 4005 (LAN) for your Virtual Machine, this can be manually be done at Network & Virtual Switch. Assigning the VLAN directly to the adapter and then you create a Virtual Switch for it to use within VM.

Though you already got it working in a different way, if I have time I will write how to install it with the QNAP TS-EC880 using the QNAP VM installer way (Manually).

Though I got to say the QNAP QGD-1600P is one interesting device, was planning to go for a QOTOM mini pc but for an i3-4005U it was £150 and then add the custom tax on top with no warranty. Decided not to as I could just recoup some cost selling my switch, as the QNAP is also PoE.

ODRIOD-H2 is also a cheaper alternative solution
https://www.hardkernel.com/shop/odroid-h2/

I don't see a problem running PFSense in VM on QNAP, as Synology, FreeNAS can so the same thing. Though you don't want to put PFSense on your main QNAP server that is expose as you would need to regularly update the firmware, which means your network goes down with it whenever you do a restart.

User avatar
Moogle Stiltzkin
Ask me anything
Posts: 9079
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: [guide] pfsense VM on QNAP in 2020

Post by Moogle Stiltzkin » Wed Jun 10, 2020 5:02 pm

in pfsense logs, i noticed a pppoe connection issues

https://www.reddit.com/r/PFSENSE/commen ... ird_issue/


my internet works... but the logs worry me :/ i don't know why this is happening...

[–]Zetto-

I had this exact problem back on 2.3.x. If you are MAC spoofing on the WAN interface try removing it.
[–]luxlucius

I'm doing MAC spoofing bc this is a new internet facing device and the filter from ISP side is by mac and username/password. Why would a mac change be a problem?

only lead i got so far is this script, how to install it, no idea.
luxlucius

So I created a script in /usr/local/etc/rc.d/ containing /sbin/ifconfig re0 promisc (tnx DutchOfBurdock) which solves the problem. But why all of a sudden pfsense started behaving like this?
hygri

Could also install the shellcmd package and set the following (assuming WAN is on re0) as an earlyshellcmd

/sbin/ifconfig re0 promisc
[–]luxlucius

I did that with a script in /usr/local/etc/rc.d/.

So far so good
[–]hygri

Sweeeeeeet. Pretty sure the shellcmd stuff is stored in the config, so it'll be handy if you ever need to restore/redeploy that config
jimp
REBEL ALLIANCE DEVELOPER NETGATE

Personally, I keep custom files under /root/ so they're easy to keep track of. Really though, as cmb said, it doesn't matter so long as you know where they are.

You can use the "Backup" package to backup arbitrary files and directories.

If you want to follow hier(7) for your scripts, then common programs and utilities go in /usr/local/bin/ and administrator and system daemons/utilities would go in /usr/local/sbin/
https://forum.netgate.com/topic/102954/ ... intenace/6

If promiscuous mode is enabled, all packets on the network will be received by the interface.
Promiscuous mode is a network card setting that does not filter incoming packets by MAC, but rather receives and accepts all incoming network data. This setting is commonly used to sniff all network traffic and to help diagnose networking issues. However, it may also be used to look for any unencrypted data such as usernames and passwords entered in FTP and Telnet.
https://www.computerhope.com/jargon/p/prommode.htm
What is a promiscuous mode for a NIC?
Promiscuous mode or promisc mode is a feature that makes the ethernet card pass all traffic it received to the kernel. It is usually used by a packet sniffing program like Wireshark, and tcpdump. If there was such program intentionally running or bridged networking for hardware virtualization, the “promiscuous mode” message might be simply ignored. Otherwise, deep investigation on that system will be required due to a security issue.

When a network card is in promiscuous mode, it can read all traffic it received rather than just packages addressed to it. Suppose for eth1, promiscuous mode is basically used to pass all traffic that ‘eth1’ receives rather than just frames addressed to it. A network card usually is in promiscuous mode when:

-If it was manually configured in that mode using ifconfig command.
-If a Network monitor tool is used, like tcpdump etc.
-In bridge network, the NIC is mostly required to operate in promiscuous mode.
https://www.thegeekdiary.com/how-to-con ... ntos-rhel/


MAC Spoofing
The MAC address field changes the MAC address used by the network card. This is usually only needed when replacing an existing device and the ARP table of connected devices cannot be controlled or easily changed.

In some cases, spoofing the MAC may require running the NIC in promiscuous mode. This is uncommon and is isolated to cases with certain network card chipsets, and certain cases when spoofing the MAC on VLAN interfaces.

In these cases, one option is to install the shellcmd package or add a command manually to run a command such as:

Code: Select all

/sbin/ifconfig em0 promisc
https://docs.netgate.com/pfsense/en/lat ... tings.html

VMware and promiscuous mode

Hi all,

Just for clarity, WHICH interfaces need promiscuous mode enabling on the vSwitch you use for pFsense? WAN? LAN? Both?
[–]ToiletDick

If you use CARP with virtualized pfsense you will need promiscuous mode. If you also have multiple interfaces on the vswitch you also need to set Net.ReversePathFwdCheckPromisc to 1 or you'll get all kinds of seemingly random connectivity problems.

If you're running a normal pfSense you do not need promiscuous mode at all.
https://www.reddit.com/r/PFSENSE/commen ... uous_mode/



bekax5 wrote:
Sat Oct 07, 2017 9:20 am
Virtual Switch (advanced functionality)

I'd like to request some "advanced" functionality to be added on the virtual switch.

Given that these are used together with the QNAP hypervisor, they should support basic network functions so that VMs could use the network as they should.

For example, there is no way to really dedicate a NIC to a VM!

More specifically, every unicast packet without the VM's IP is dropped. And why does this happen if the VM has no IP since it's in promiscuous mode? It should receive in it's interface everything that has been sent to the QNAP physical interface!

Thus, the VM can't correctly use promiscuous mode, because the virtual switch drops the packets!


I believe the correct network mode should be "external-mode" according to QNAP. However this isn't in fact a dedicated mode since the QNAP vswitch controller appears to give instructions to drop packets whenever it "thinks" that the IP isn't inside of the QNAP.


So either:
- Fix the External-Mode, so it doesn't drop packets (this should be dealt by the VM and not the virtual switch with IP 0.0.0.0)
- Add support for Promiscuous Mode
- Add support for VM dedicated NIC



These are some really basic functions that I can't believe aren't yet working on QNAP devices!
It's crazy to thing there is a hypervisor but there's no way to force traffic to a VM or an interface!
viewtopic.php?t=136288


Virtual Switch Setup for Pfsense VM
by bekax5 » Dec 25, 2017

I had some issues a few months ago with similar setup.
I ended up realising that in the new QTS it's impossible to have dedicated interfaces as opposed to what says in the website and as opposed to previous QTS.

It appears that every interface now must pass through a virtual switch and thus not allowing all traffic to flow into the VM.
What would be needed for that is either a promiscuous mode, or a dedicated NIC for the VM.

Unfortunately even after speaking with one of their senior engineers I don't believe they really understand the repercussions in that.


From my point of view they believe what I complained is just a singular use case, and not every user that wants to have VMs with firewall, routing, switching, pfsense, etc...

I guess more people should complain and ask to add a dedicated NIC option for VMs.

More on that in the previous thread I opened requesting "new features" that are in fact removed features.
viewtopic.php?f=24&t=136288
viewtopic.php?t=135574



re0 / em0 and ect, what are the differences ?

In FreeBSD the devices are name after the driver name. So re0 means that you have a RealTek 8139C+/8169/816xS/811xS/8101E PCI/PCIe Ethernet adapter. em0 means that you have a Intel(R) PRO/1000 Gigabit Ethernet adapter driver.

Have a look at each driver man page.
re(4)
em(4)
https://forums.freebsd.org/threads/re0- ... nces.5494/


afaik the solution for a normal pfsense vm setup would probably be either

Code: Select all

/sbin/ifconfig re0 promisc

Code: Select all

/sbin/ifconfig em0 promisc

but when adding qnap virtual switch into the equation, i'm not sure that this is a good thing to do :shock: i already asked helpdesk for help on this since i don't know what to do at this point to fix it.
Last edited by Moogle Stiltzkin on Fri Jun 26, 2020 7:47 am, edited 1 time in total.
NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[Backup] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) single disks.
[^] QNAP TS-659 Pro II
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228
[^] QNAP TS-128
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Win10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin

User avatar
Moogle Stiltzkin
Ask me anything
Posts: 9079
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: [guide] pfsense VM on QNAP in 2020

Post by Moogle Stiltzkin » Wed Jun 10, 2020 6:22 pm

Yippym wrote:
Wed Jun 10, 2020 4:44 pm
Moogle Stiltzkin wrote:
Wed Jun 10, 2020 9:25 am
i'll check that review out :'

it has a qunet switch hm. vminstaller? that might be unique for your model, cauz i didn't see that for my ts-877 in appcenter.
Ah, I assume the VM Installer is available for all.

All it does is assigned a VLAN 4001 (WAN) and VLAN 4005 (LAN) for your Virtual Machine, this can be manually be done at Network & Virtual Switch. Assigning the VLAN directly to the adapter and then you create a Virtual Switch for it to use within VM.

Though you already got it working in a different way, if I have time I will write how to install it with the QNAP TS-EC880 using the QNAP VM installer way (Manually).

Though I got to say the QNAP QGD-1600P is one interesting device, was planning to go for a QOTOM mini pc but for an i3-4005U it was £150 and then add the custom tax on top with no warranty. Decided not to as I could just recoup some cost selling my switch, as the QNAP is also PoE.

ODRIOD-H2 is also a cheaper alternative solution
https://www.hardkernel.com/shop/odroid-h2/

I don't see a problem running PFSense in VM on QNAP, as Synology, FreeNAS can so the same thing. Though you don't want to put PFSense on your main QNAP server that is expose as you would need to regularly update the firmware, which means your network goes down with it whenever you do a restart.
that is interesting. because i was trying to get quwan to work first. but my isp requires vlan tagging for the internet and iptv (although you can ignoring setting up for iptv vlan tag if you aren't using it). how to go about setting the vlan tagging for the virtual switch, or within quwan? i pointed this out to qnap helpdesk and they said they will look into it. It was pretty odd, like with pfsense they don't force you to have internet connectivity to setup and run pfsense web ui. once you are in pfsense web ui, you can then configure the WAN at that point, which for my case i had to first configure the vlan tagging for the PPPOE for that to work. By comparison, Quwan just locks you out entirely because i could not connect to PPPOE, because a vlan tagging for pppoe was not in the settings.

in pfsense i had to vlan tag then bind to pppoe. not sure that is the same thing as vlan tag the physical or virtual ports (e.g. virtual switch management in qts), is it? :'

i noticed in physical switch, there is indeed vlan tagging. but when i tried that, it didn't seem to work for me. is it because i setup a virtual switch to bind to that physical port? no idea. only vlan tagging within pfsense worked.

anyway i'm pretty sure i could not find "VM Installer" for my normal nas in appcenter. i suspect that is a feature only for your router + switch + nas model aka QGD. At most we only got virtual station for vms on the qnap.
NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[Backup] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) single disks.
[^] QNAP TS-659 Pro II
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228
[^] QNAP TS-128
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Win10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin

User avatar
Moogle Stiltzkin
Ask me anything
Posts: 9079
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: [guide] pfsense VM on QNAP in 2020

Post by Moogle Stiltzkin » Wed Jun 10, 2020 7:59 pm

Trexx wrote:
Tue Jun 09, 2020 11:06 pm
...
request completed 8)
NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[Backup] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) single disks.
[^] QNAP TS-659 Pro II
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228
[^] QNAP TS-128
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Win10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin

Post Reply

Return to “Users' Corner”