Duplicate File Finder (File Comparrison, File Compare)

Tell us your most wanted features from QNAP products.
smitty123

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by smitty123 »

it's real, but your suggestion is interesting

looking at https://www.qnap.com/i/useng/app_center ... jump_win=1 it says linux-station needs 4gb ram.

so looks like it won't work on my nas (arm, 256mb ram).

still hoping someone at Qnap will port this in QTS since it's also a linux base.
User avatar
Don
Guru
Posts: 12289
Joined: Thu Jan 03, 2008 4:56 am
Location: Long Island, New York

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by Don »

Spam meaaages deleted
Use the forum search feature before posting.

Use RAID and external backups. RAID will protect you from disk failure, keep your system running, and data accessible while the disk is replaced, and the RAID rebuilt. Backups will allow you to recover data that is lost or corrupted, or from system failure. One does not replace the other.

NAS: TVS-882BR | F/W: 5.0.1.2346 | 40GB | 2 x 1TB M.2 SATA RAID 1 (System/VMs) | 3 x 1TB M.2 NMVe QM2-4P-384A RAID 5 (cache) | 5 x 14TB Exos HDD RAID 6 (Data) | 1 x Blu-ray
NAS: TVS-h674 | F/W: 5.0.1.2376 | 16GB | 3 x 18TB RAID 5
Apps: DNSMasq, PLEX, iDrive, QVPN, QLMS, MP3fs, HBS3, Entware, DLstation, VS, +
User avatar
OneCD
Guru
Posts: 12155
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by OneCD »

@smitty123: a couple of posts were removed from this topic as they were in breach of the forum rules regarding advertising - and they fit the spambot profile. ;)
[b]QNAP admin[/b] wrote: By participating in the Forums, you agree not to:
  • post any spam messages.
  • post advertisements, chain letters, junk email or bulk communications.

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage
Blisk
Know my way around
Posts: 143
Joined: Fri Jul 08, 2011 5:54 am

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by Blisk »

after two years still no software for duplicated files?
User avatar
Trexx
Ask me anything
Posts: 5388
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by Trexx »

There are many way to accomplish this (dupeguru in Docker, Qfiling, VM options, Entware, etc.)
Paul

Model: TS-877-1600 FW: 4.5.3.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x1TB SK Hynix Gold
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
UPS: CP AVR1350

Model:TVS-673 32GB & TS-228a Offline[/color]
-----------------------------------------------------------------------------------------------------------------------------------------
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq
User avatar
parkerc
Easy as a breeze
Posts: 314
Joined: Thu Oct 16, 2014 7:36 am
Location: Near London, England
Contact:

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by parkerc »

This maybe one of my old posts, but I thought I’d update it to say that I’m currently using fdupes to find duplicate files. I have it installed as a .qpkg thanks to the QNAP club.

https://www.qnapclub.eu/en/qpkg/418

You can tweak the published .php file to change the command line used. I changed mine to ignore hidden files (as QNAP has so many) and group by size.

Code: Select all

 fdupes -ASr /share > /share/Public/fdupes.output
Model Name : QNAP TS-253be (16GB) & TS-453D (16GB)
Firmware Version:: QTS Latest
OS Version : Apple iOS (iPad/Safari Browser)
Number & Type of HDDs : 2 x 3TB WD Reds, Raid 1 & 4 x 4TB WD Reds, Raid 5
Website (When I have time) : http://www.nodecentral.co.uk
shooms99
Starting out
Posts: 23
Joined: Thu Aug 11, 2016 12:44 pm

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by shooms99 »

Trexx wrote: Tue Feb 27, 2018 12:00 am There are many way to accomplish this (dupeguru in Docker, Qfiling, VM options, Entware, etc.)
How do I use QFiling to look for duplicate files? I don't have a specific file in mind, but I just want to generate a list of all ducplicate documents (photos, videos, files, etc.) so I can free up space on the QNAP.
User avatar
Xoffis
Starting out
Posts: 10
Joined: Fri Nov 29, 2019 9:01 pm
Location: Sweden
Contact:

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by Xoffis »

I also installed QFiling/Qsirch just for dupe search, don't seem to have that function?

So I do want to +1 on this!
Sorry, English is not my native language, plz have oversight
And i have autism, so try not to answer with irony and so on as i can have large trouble understand it and may take it personally

A NAS n00b with my first real NAS a TS-231P2
4Gb RAM
2x6TB IronWolf drives
shooms99
Starting out
Posts: 23
Joined: Thu Aug 11, 2016 12:44 pm

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by shooms99 »

If you are comfortable with SSH, I was able to follow this walkthrough as a guide:

https://blog.linuxserver.io/2017/09/17/ ... s-on-qnap/

instead of using TVHeadEnd, I chose a random dupeguru image from the repository, which was created by another user.
User avatar
Moogle Stiltzkin
Guru
Posts: 11445
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by Moogle Stiltzkin »

shooms99 wrote: Thu Dec 12, 2019 8:08 pm If you are comfortable with SSH, I was able to follow this walkthrough as a guide:

https://blog.linuxserver.io/2017/09/17/ ... s-on-qnap/

instead of using TVHeadEnd, I chose a random dupeguru image from the repository, which was created by another user.
i couldn't figure out how to get dupeguru docker running, so i instead ran dupeguru from windows pc and pointed to the QNAP share and that seems to work, albeit there is the 1gbe bottle neck :(
https://dupeguru.voltaicideas.net/

I guess using win10 Virtual station and using dupeguru from there works as well, that part i know how. but if all you require is just dupeguru, then docker is much more efficient if you don't require full blown vm.
NAS
[Main Server] QNAP TS-877 (QTS) w. 4tb [ 3x HGST Deskstar NAS & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A (Truenas Core) w. 4x 2TB Samsung F3 (HD203WI) RaidZ1 ZFS + 8gb ddr3 Crucial
[^] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) 2x 4TB Seagate Ironwolf, Raid5
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-253D (Truenas Scale)
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Qotom Pfsense|100mbps FTTH | Win11, Ryzen 5600X Desktop (1x2tb Crucial P50 Plus M.2 SSD, 1x 8tb seagate Ironwolf,1x 4tb HGST Ultrastar 7K4000)


Resources
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin
User avatar
storageman
Ask me anything
Posts: 5506
Joined: Thu Sep 22, 2011 10:57 pm

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by storageman »

Maybe Hero will be available for you guys
rober1967
First post
Posts: 1
Joined: Wed Jun 14, 2017 4:33 pm

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by rober1967 »

Hi, I've a ts251. Is there some tool that can be installed on nas and that can list all duplicated files ? I installed qfiling, qsirch, qumagie, but I didn't got how to use to search duplicate.
Thanks.

Robert.
plip1978
New here
Posts: 4
Joined: Mon Feb 15, 2016 3:02 am

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by plip1978 »

+1 on this too. Especially for photos!
Locked

Return to “Features Wanted”