[Megathread] Moogle's QNAP FAQ help

[Johnno72]

have you tried using a convertor to reduce the size? I see the real problem is the 96x96 size limit I quickly found and tried these on your avatar
http://www.shrinkpictures.com/create-avatar/resize.php
http://picasion.com/get-avatar/

[Moogle Stiltzkin]

have you tried using a convertor to reduce the size? I see the real problem is the 96x96 size limit I quickly found and tried these on your avatar

thx those are great *yoink

i wish the forum supported sizes like this *a buddy made it for me back in the day



also wanted to add a nicer megathread banner using this but, seems too big
http://i.imgur.com/ZrG2ZCO.png

Hiya Moogle, I think it would be that the information they wanted wasn't in here at the moment as you said. You are doing well in my opinion for someone dedicating their time to research and create these documents and information for free to others in the community.

I hope you can see whom down voted your free information so you can find out why.

np i don't take it personally (not gonna point fingers, though do have some prime suspects ), though i do hope it's at least constructive criticism and not just trolling (i would like to believe our qnapers have better things to do ) could always use feedback to improve the thread. too bad we don't have private messaging

[Moogle Stiltzkin]

swap memory usage is too high, what does this mean?

I saw problem threads like this in this link below. Normally when you get a "memory usage is to high" warning, what it essentially means is, you ought to upgrade to more ram. 4-8gb is usually a good place to start. some people go straight to 16gb if they really need that much.
viewtopic.php?f=25&t=132156

[youtube=]jZYPxosPKA4[/youtube]

[youtube=]p3q5zWCw8J4[/youtube]

How much ram do you need for your QNAP nas ?
[youtube=]dR4e4cn5U8o[/youtube]

Also if you intend to use Qsirch, from my own testing, 2gb ram is able to but barely enough to Qsirch. At the very least it leaves little ram for much else than Qsirch and storage.
viewtopic.php?f=24&t=134081#p625242

32-Bit vs 64-Bit - The Advantage

[youtube=]KgiMzKb8dD0[/youtube]

[Moogle Stiltzkin]

How do hard drives work?

[youtube=]wteUW2sL7bc[/youtube]


http://www.anandtech.com/show/9858/seag ... r-20-years



http://www.anandtech.com/show/9866/hard ... ve-in-2018



https://www.extremetech.com/computing/1 ... hard-drive

[aspomwell]

I agree with Johnno. Awesome thread.

"some people just want to watch the world burn"

[Moogle Stiltzkin]

I agree with Johnno. Awesome thread.

"some people just want to watch the world burn"

thx

[Moogle Stiltzkin]

Cloud backup what is it?
[youtube=]DTbil6GaLIs[/youtube]

What is the cloud in general?
[youtube=]dsKIpLKo8AE[/youtube]

Cloud Storage vs Cloud Backup – What's the Difference?
[youtube=]DZ1UK1dRnSo[/youtube]

Local Backup vs Cloud Backup: What's the right choice?
[youtube=]tqQyfJXTLRk[/youtube]

A simple video guide what to look for in cloud backup service
[youtube=]3A3jNIdCrSM[/youtube]

personally these are the main points that interest me most

- is backup encrypted? (i'm never gonna use cloud unless it's encrypted at the storage location)

- what is privacy policy for data in the server host country? (like i would never use a server in North Korea for example. Switzerland or France seem like better ideal places because they have strong privacy laws, but the server location in relation to you may impede upload performance is the only downside perhaps?)

- network quality ( specifically upload and download. If the speed is too slow may not be worth my time trying to sync to backup to that cloud service)

- price? for what storage capacity? any major caveats for the package? ( the top of the list is usually cloud services like crashplan which cost only $5 usd monthly for unlimited storage (though technically your limited by your NAS ram, storage capacity, broadband upload speed); hubic is almost similarly priced, with also a big storage allowance)

when accounting for price when comparing NAS to NAS backup vs NAS to cloud, keep in mind running a NAS also incurs an annual operating cost such as electricity bill, not to mention wear on HDD which usually after 5 years requires replacing. Not to mention that any technical failure falls onto the operator to deal with any disaster (like raid, hdd failure, theft, fire etc....) So when you take that into account 5usd monthly for unlimited storage for an off site backup to the cloud where IT experts manage on your behalf to keep your backup storage safe, secure, available when you need it is not such a bad deal.

NAS to NAS is still a valid backup option in terms of backup speed performance especially if you have don't have good broadband speed to backup remotely online. Or you want a more reliable 1 2 3 backup which is a combination of 2 NAS + 1 cloud backup usually. But most people usually opt for a 1 2 approach either NAS (your new model) to NAS (to an older model relegated as a backup) or NAS to cloud.

recommendations

also when i'm talking about cloud backup, i'm not talking about the over priced with very limited storage packages out there. the target audience for those packages are for low data storage users on the desktop using 1tb or less (their business model is to price gouge on these types of users). The one i'm most interested in is full NAS backup which usually means somewhere between 10-30tb backup for something like 5 USD monthly

The cloud backups most popular with that in mind for QNAP is crashplan first and fore most (use either Dynek's crashplan QPKG, or search forum for setting cp up using docker). Hubic also is another good alternative (hybrid backup sync supports this, or stephs qpkg). If you know another great option do share with us


bottomline
I recommend either NAS with cloud backup (great for mobility/flexibility/less hassle for technical maintenance for backup). Or NAS to NAS (best backup performance speed, more control of data/privacy) for your backup solution depending on your circumstances.

[Moogle Stiltzkin]

Dummies guide for understanding the current SSD tech trend leading up to 2017

Flash Industry Trends Could Lead Users Back to Spinning Disks
by Chris Ramseyer June 9, 2017

If you already own an SSD produced in the last five years, you may want to hold onto it. Next generation products will shoot for the moon on paper but fall well short of leaving the atmosphere. For the last decade we've watched the technology progress, but at the same time, meaningful growth has slowed or stalled due to cost-saving measures in NAND and controller technologies.

Early adopters, along with data centers and enterprise customers, played a large role in funding nearly all of the last decade's NAND development. The decision to purchase these products was due to how dissimilar they were from HDDs in terms of performance and reliability.

Yet over the last two years, the trend has been to slow performance to reduce costs. The more the technology is neutered, the closer to hard disk performance we see. On the controller side we've seen the number of processor cores and channels from each controller to the NAND flash shrink. On paper the new flash is faster than the old flash, so it's possible to achieve the same performance with fewer channels, but the larger die sizes also give us less parallelization. On the flash side, the move to more cost efficient 3-bit per cell (TLC) has delivered less sustained performance for heavy workloads that take longer to complete. Those are the same workloads early adopters chose flash in the first place.

SLC buffers, user-data in DRAM, and other technologies have hid many of the shortfalls of modern low-cost SSDs from many people, but power users can spot the difference between older and newer products. During this time, you still had an option to purchase higher cost MLC-based SSDs even though the number of products had declined. In the trailing end of 2017, however, those products will virtually disappear and you'll be forced to seek out alternatives.

full article:
http://www.tomshardware.com/news/consum ... 34631.html

If you haven't been keeping up with tech, this is definitely a good read especially for understanding ssds and whats going on with the current trend

but bottom-line for me when it comes to ssd *speaking from a desktop user standpoint

- M.2 NVME (no bottle necks)
- price? (i'm more closer to budget bang for buck oriented, i only pay for performance if it has good value)
- endurance (love my samsung 850 pro if other ssd has similar or better endurance to it that would be great, but for me this is the gold standard of endurance i'd expect from an ssd)
- storage capacity (depends what you want to use them for. i consider ssd premium storage for OS and other regularly used apps, for desktop use i'm fine with 500gb, but i can definitely go for a 1tb if it's within budget. Everything non important can dump on cheaper 4tb hdd if they don't need such fast access latencies as much)

[Moogle Stiltzkin]

Too often i see qnap users get a model for media server for heavy use but end up getting the wrong model for the intended workloads. So hopefully these articles help to educate others in picking the right qnap model within your budget to get the job done. It's a long read but it's worth reading

Building a Docker Media Server – Introduction
FEB 2, 2017 by Terry Walsh



That one-box media server – a blend of low-power, high-capacity and ultimate flexibility became real at last. For a time, this became my media server setup. I ran a QNAP NAS with a Plex Media Server package alongside a Windows 8 (at the time) VM running those same supporting applications. I switched off that dedicated Windows PC I has running 24/7 and that was a good job done.

Or so I thought.

In fact, I grew frustrated with this configuration pretty quickly. The problem? Performance. As those of you that have dabbled with VMs will be aware, running guest operating systems can use up a lot of resources on the host. Add an increasing number of media apps to that guest OS and, before you know it, the host is going to struggle unless it’s equipped with some serious power.

One of the major benefits of running a Linux-based device is its low power requirement. Linux benefits from a small footprint, hence the plethora of cheap, low-power (performance as well as wattage) NAS devices available out there. For simple file serving duties, a humble engine is fine. But virtualization needs a lot more power. Combine that with heavier native workloads – like real-time media transcoding – and you can understand why QNAP introduced supporting powerhouse NAS systems with Intel Core i7 processors and 8 or 16 GB RAM!

Those systems, like the QNAP TVS-871 (which I still use today) or the newer TVS-x82 Series are really closer to small business servers than consumer NAS devices. Under load – which even these mighty systems are when combining media server and hypervisor duties – the question is whether you’re really reducing power consumption compared to a two-box solution?

A New Approach

It was time to consider a new approach. One that could deliver more of the features I really needed from a media server.

Simplicity – a single-box system that’s easy to build, configure and manage without the need for a PhD.
Compatibility – support for all of the major media server apps and services. Platform agnostic, ideally.
Flexibility – as a serial twiddler, it should be easy to spin up and test new or emerging applications and services (and get rid of them quickly if they don’t work out).
Reliability – at the core, the media server needs to be built on a strong, stable foundation that performs well and won’t fall over.
Frugality – a compact system (both in physical footprint as well as resource consumption) that could draw on reserves when needed, but otherwise remained nimble with low running costs.

Reading around, it quickly became clear that containers could be the answer.

Introducing Docker and Docker Containers

In short, I was looking for a way to deliver the same flexibility that my existing QNAP solution delivered – so some degree of virtualization would be required – but with a much lower performance overhead. This is the major benefit of containers (also referred to as containerization or container-based virtualization).

Whereas “traditional” virtualization works at the operating system level (for example, running a full-blown instance of Windows 10 on that QNAP NAS I referred to earlier), containers work at the application level. They allow an application to run virtually without the need to launch a supporting guest operating system (and all of the bloat that goes with it).

Containers are isolated from the host operating system, (they actually run on a single, lightweight Linux engine) and include the files, environment variables and libraries required to run the application. However, they share other resources required from the host operating system – binaries, libraries as well as physical resources. That means a containerized app runs much more efficiently than if it was installed in a virtual machine, running on a host operating system.

read the full article here
http://wegotserved.com/2017/01/31/build ... roduction/

Building a Docker Media Server – Hardware Selection
FEB 2, 2017 by Terry Walsh

If you’re thinking about using an off the shelf QNAP NAS device , then you’ll definitely need to take a step back and consider your needs before wading through the plethora of devices available. But, before we get too specific, let’s talk more broadly about media server hardware.

The developers of Plex recommend a processor that has a PassMark score of at least 2000 for transcoding a single 1080p high definition file, encoded at a 10 Mbps bitrate (decent quality, but not excessively high). For 720p/4Mbps files, the figure drops to a PassMark of 1500. That’s for a single, transcoded stream. Where no transcoding is required (local playback, or where your client device supports the filetype) you’ll be fine with a lower specification CPU. Want to transcode multiple streams simultaneously? Multiply that PassMark figure by the number of streams for a rough estimation of performance.

I bet you’re wondering what the PassMark score is for your media server now, right? I have two options for you: first, you can refer to the http://cpubenchmark.net/ website to search previously tested results for a swathe of processors. Alternatively, download a trial edition of the excellent PassMark PerformanceTest (http://www.passmark.com/products/pt.htm) benchmark suite (which we use for most of our reviews at We Got Served) which not only will test your home server’s CPU and graphics capabilities, but can also be used to test your network speeds (more on this here: http://www.passmark.com/products/pt_advnet.htm).

As an example, I’ve run PerformanceTest on a media server I’m using at the moment. It’s running an Intel Core i5-4790 processor (getting a little old now, but still very capable). It gets a CPU PassMark of 7471, meaning (again, roughly) I should expect to be able to transcode three simultaneous 1080p streams on this media server. That’s going to work well for my family.

To put that result into context, I also ran the benchmark on a PC running a low-power mobile Intel Core i5 processor (the 1.3 GHz Intel Core i5-4250U) and the result was just 3400 – good enough for a single stream, but that’s it. At the other end of the spectrum, an Intel Core i7-6700K picks up a PassMark of 11327.

As an aside, I’m basing this discussion around Plex Media Server, but that’s only part of the story. At the time of writing, Plex Media Server transcoding does not take advantage of supporting media encoding technologies like Intel Quick Sync Video (built into many Intel CPUs) and Nvidia NVENC (supported on the company’s graphics cards). So it requires the CPU itself to do the heavy lifting. Other media server apps, like Emby, are able to offload transcoding to Quick Sync Video/NVENC so may facilitate a (slightly) lower specification. Plex currently has hardware transcoding support in beta, so it looks like its on the way for 2017.

By now, I’m sure you get the point. Sure, that ARM CPU and 2 GB RAM will deliver the basics of a competent media server. But if you want to do more, you’re going to have to spend more. My experience is that once you’re up and running, you always want to do more, so it makes sense to specify hardware for tomorrow’s requirements, rather than today’s!

read the full article here
https://wegotserved.com/2017/02/02/buil ... selection/

Building a Docker Media Server – Configuring Your Network (Part 1)
FEB 9, 2017 by Terry Walsh

http://wegotserved.com/2017/02/09/build ... -part-1/3/

Building a Docker Media Server – Configuring Your Network (Part 2)
FEB 13, 2017 by Terry Walsh

https://wegotserved.com/2017/02/13/buil ... rk-part-2/

Moogle's 2017 qnap plex recommendation

In my own opinion i feel that something like these would be a great place to start as a baseline recommendation, a TVS-473 or a TVS-682 Though check if there are newer models with better specs and pricing than these model, cause i don't keep track of that to update this as it's mostly just a baseline recommendation example for 2017.
[youtube=]LkYmYY74p1Q[/youtube]



Intel Core i3 model, the TVS-682 is definitely no budget media server, but should provide strong performance and expandability that’ll last for years to come.
[youtube=]yevprZzm16w[/youtube]



Another good recommendation is the TVS-473 (which has a AMD RX421BD Quad-core 2.1GHz cpu) from the TVS-x73 series which consists of TVS-473/673/873.
[youtube=]MhAujKU1eXs[/youtube]


you can also see further additional reading on this subject here as well on
(page1) *which also includes plex googlesheet for qnap recommended models for transcoding

Code: Select all

QNAP Media playback options

[Moogle Stiltzkin]

What is BTRFS?

Alrighty then ....

Many on this forum seems to have forgotten that the very purpose of BTRFS is that it unlike other filesystems will regurgitate your data if it ate it.
Yes, there are horror stories about people loosing their filesystem and for something that WAS experimental this is very likely to happen anyway.
After about kernel 4.4 or so things started to really get in good shape. Now certain features of BTRFS is working quite well and most of the focus now are to fix things instead of implementing shiny stuff.

I have used BTRFS for years now as a simple replacement for ext4. I do not use any of the fancy features of BTRFS like subvolumes, snapshots etc. What I care about is the redundancy features in BTRFS that make sure that the data you put in is the data you get out. If you educate yourself a bit (and are a bit paranoid) you will soon find out what configuration are safe to use or not.

Basic BTRFS usage is in many ways better than a "regular" filesystem like ext4 for example. Why ? simply because if you corrupt something in ext4 or xfs then yes, the filesystem structure may be repairable, but the data will be corrupt. BTRFS will thanks to checksumming all data report back what file was chewed on and if you are lucky enough to run a redundant configuration like DUP, RAID1, RAID10 then BTRFS will fix your broken data.

Now, md raid (mdadm) is rock solid and very reliable. I have not read any horror stories about people loosing an entire array unless they seriously does not know what they are doing. md raid simply works and is fantastic for what it is designed for. Now there are some drawbacks, but that is not any fault of md raid.

You need disks of the same size, raid is designed to protect against disk failure (not bit-rot)
raid5 will detect a bit failure, but it will not be able to correct it since it does not know what drive the corruption originated from.
If the entire disk where the corruption originated from is lost , then ironically you are able to restore your lost data since you know what data (disk) is bad.

By the way ... raid5 works by XOR'ing data for those who does not know (see here for a great little explanation of how it works : http://blog.open-e.com/how-does-raid-5-work/ )

What btrfs does is that it checksums every piece of data it write. So if it need to write "hello world!, how are you today" it will add a little bit more information so it will know if the data is correct or not. This means that btrfs will always know what data have been chewed on and therefore can correct it by either using a good copy , or using good parity for reconstruction. BTRFS does NOT yet compute checksums for it's parity , however md raid does (to my knowledge) not do that either so in principle it is just as safe. ( I am not saying that you should use BTRFS RAID5/6 for now, it still have a long way to go ).

Both btrfs and md raid is deterministic. e.g. it will only read data once of the drive(s) it needs to read from. This mean that neither md raid or btrfs will verify that redundant data is correct on read (unless the read fails in which case the other copy will be tried out). Now let's pretend that copy B is broken, if data is read from copy A always you would now know that copy B is broken UNLESS copy A is gone toast. Therefore you have to periodically run a scrub regardless if it is btrfs or md raid to make sure that you data is in good shape.

The difference here is that btrfs will detect data corruption while md raid will not.

That being said , my most important data is on md raid 6 with ext4 as the filesystem. The only reason I have not migrated to BTRFS yet is that BTRFS' raid 5/6 is unstable and not well tested. When BTRFS raid5/6 gains more traction I will be happy to migrate one of our systems to btrfs metadata raid1 or raid10 and data raid 5/6. Why? well, metadata is data about the filesystem and raid1 is stable. I am willing to risk loosing a file or two as long as the filesystem structure itself is intact.

A last word of advice is - follow the BTRFS mailing list. There you will see that those who run into problems either run on the latest shiny stuff, have weird configurations, bad hardware or does not understand how things work. The short version is : BTRFS is in fact better than you think it was

- waxhead

[youtube=]ISG_WphoAjo[/youtube]

For news updates regarding BTRFS i usually check here
http://www.phoronix.com/scan.php?page=s ... Btrfs+RAID

BTRFS is an awesome filesystem but the raid is not prime for any production environment (specifically for it's raid5/6), but moving forward it's the filesystem of the future for sure (once it's ready)

[Moogle Stiltzkin]

Why 10 Gigabit Ethernet? (Introducing 10 gigabit)

[youtube=]orUZMvxWrAI[/youtube]

What is SMB Multi-Channel? how does it help increase network speed throughput?

[youtube=]0rFL8PGkmMU[/youtube]

To distill it down...

Link Aggregation will only increase aggregate bandwidth, it does NOT increase single user bandwidth one iota.

You'll have to rely on SMB Multichannel to get higher single user bandwidth. This does NOT require a managed switch.

It also only works with SMB. If you are connecting to the server through a different means, it does not increase bandwidth (the good news is, if you are attempting to load a file from a network share in just about any program, it is connecting through SMB, unless of course you are using an iSCSI or NFS share, which is not a default network share configuration in windows).

The other downside is that most laptops have a single network ports, which means SMB Multichannel won't do a thing for you. However, if you can add on a second port through some means, that might work for you. I haven't tried it and they are RARE, but a dual port USB3 network adapter MAY work with SMB Multichannel. It also may not. SMB Multichannel can be slightly finicky from what I have found (best odds of working, same adapters. I have NOT been able to get it working with a PCIe NIC combined with a USB3 NIC).

Benefits

SMB Multichannel allows file servers to use multiple network connections simultaneously and provides the following capabilities:

- Increased throughput. The file server can simultaneously transmit more data using multiple connections for high speed network adapters or multiple network adapters.

- Network Fault Tolerance. When using multiple network connections at the same time, the clients can continue to work uninterrupted despite the loss of a network connection.

-Automatic Configuration: SMB Multichannel automatically discovers the existence of multiple available network paths and dynamically adds connections as required.

Requirements

SMB Multichannel requires the following:

1. At least two computers running Windows Server 2012 or Windows 8.
2.At least one of the configurations below:

Multiple network adapters
-One or more network adapters that support RSS (Receive Side Scaling)
-One of more network adapters configured with NIC Teaming
-One or more network adapters that support RDMA (Remote Direct Memory Access)


Installing

SMB Multichannel is enabled by default. There is no need to install components, roles, role services or features.
The SMB client will automatically detect and use multiple network connections if a proper configuration is identified.

https://blogs.technet.microsoft.com/jos ... d-smb-3-0/

does Smb multichannel work on qnap?

I will type up more later when I have more time and list everything I did, but this is with our offices TS-1270U-RP.

Using the 2 built in 1gb NIC's on the QNAP and it's connected to a crappy Windows 10 box with dual 1gb NIC's. But yeah I got SAMBA multi-channel working on the QNAP.

so you plug in 2 ethernet ports on both ends each. Is that all? or did you need to do anything in qts settings for this to work?

You need to configure server multi channel plus some in the SAMBA config of course. Still, it's an experimental SAMBA feature. It works for me, too - but not sure it's ready for prime time.

viewtopic.php?f=151&t=129744&p=630124#p595250

How to use smb 3

[youtube=]pHxVvxj2yh0[/youtube]

https://www.qnap.com/en/how-to/tutorial ... in-qts-4-2

[MrVideo]

Ctl + mouse scroll for browser window sizing is browser specific. It is not universal.

[Moogle Stiltzkin]

Ctl + mouse scroll for browser window sizing is browser specific. It is not universal.

yep referring to browser only ^^; but i'll clarify nonetheless thx.

[MrVideo]

-Use mouse scroll + ctrl to adjust browser window to a comfortable size for easy reading when using web browser

Not any better. Browsers are web browsers. A better statement is:

-Use mouse scroll + ctrl to adjust browser window to a comfortable size for easy reading (browser specific: may not work with your browser)

[Moogle Stiltzkin]

-Use mouse scroll + ctrl to adjust browser window to a comfortable size for easy reading when using web browser

Not any better. Browsers are web browsers. A better statement is:

-Use mouse scroll + ctrl to adjust browser window to a comfortable size for easy reading (browser specific: may not work with your browser)

*updated