Sometimes in everyday computer activities the task is to find duplicate files. There can be many reasons for this: lack of hard disk space, attempts to reduce the entropy in your files, deal with photos dumped at different times from the camera, and many other necessary cases.

On the web you can find a large number of programs that allow you to search for duplicate files. But why look for some programs if a smart tool for such work is usually always at hand. And this tool is called Total Commander (TC).

In this article, I will show all methods based on Total Commander versions 8.5 , in this version, the search for duplicate files has become very rich in functionality.

!!!Small important digression. What is meant by the word duplicate file? Two files are IDENTICAL only when they exactly match bit by bit. Those. any information in a computer is represented by a sequence of zeros and ones. So, files match only when they completely match the sequence of zeros and ones that these files consist of. All talk about the fact that you can compare two files on some other basis is deeply erroneous.

TC has two essentially different methods for finding duplicate files:

• Synchronize directories;
• Search for duplicates;

Their features and applications are best illustrated by examples.

1. Synchronization of directories.

This method is used when you have two compared folders that have an identical structure. This usually happens in many cases, here are a few of them:

• You regularly backed up your working folder. After some time, you need to find out what files have been added or changed since the archive was created. You unpack the entire archive into a separate folder. The folder structure in it practically coincides with the working one. You compare two folders "original" and "restored from the archive" and easily get a list of all changed, added or deleted files. A couple of simple manipulations - and you delete from the restored folder all duplicate files that are in the working one.
• You are working in a folder on network drive and regularly make a copy to your local drive. Over time, your working folder has become quite large and the time spent on a full copy has become very large. In order not to copy the entire folder each time, you can first compare with the backup and copy only those files that have been changed or added, and also delete files in the backup folder that have been deleted from the main folder.

Once you get used to the power of this method, you will be able to come up with thousands of situations where the directory synchronization method will be of great help to you in your work.

So, how does everything work in practice. Let's get started.

Suppose we have a main folder "Working", which contains the files that are being worked with. And there is a folder "Archive", which contains the old copy of the folder "Working". Our task is to find duplicate files in both folders and remove them from the folder "Archive".

Open TC. Open the compared folders in the right and left panes:

Press menu "Commands" - "Sync directories ..."

The directory comparison window opens.

Next, we need to set the comparison parameters. Put checkboxes in the options "with subdirectories", "by content", "ignore date"

• "with subdirectories"— files in all subdirectories of the specified folders will be compared;
• "by content"- here is the key option that makes TC compare files BITS!!! V otherwise, files will be compared by name, size, date;
• "ignore date"- this option makes TC show different files, without trying to automatically determine the direction of future copying;

!!! Only files with the same name will be compared!!! If the files are identical, but they have a different name, then they will not be compared!

We press the button "Compare". Depending on the size of the files, the comparison can take a very long time, don't worry. In the end, the comparison will end and the bottom status line (section 1 in the figure) will display the result:


If the buttons in the "Show" section (section 2 in the figure) are pressed, then you will see the result of the comparison for each file.

- this button enables the display of files that are in the left panel, but which are not in the right;

- this button enables the display of identical files;

— this button enables the display of different files;

- this button enables the display of files that are in the right panel, but which are not in the left;

If you initially have all the display buttons turned off, then the result of the comparison can only be evaluated by the status bar (section 1 in the figure above), in this case we see that 11 files were compared, of which 8 files are the same, 2 files are different, and there is also a file in the left panel that is not in the right panel.

To complete our task, it is necessary to leave the display of only identical (identical) files, so all other display buttons are turned off


Now we have only identical files left, and we can safely delete them in the folder "Archive". To do this, select all files. The easiest way to do this is by pressing the universal combination CTRL+A. Or first select the first line with the mouse, then press the key on the keyboard SHIFT and without releasing it, select the last line with the mouse. As a result, you should get something like this:

In the final step, we right-click on any line and select the item in the menu that opens. "Delete Left"

TC kindly asks us again about our desire,

and if we click "YES" then it deletes all marked files in the folder "Archive".

After that, the two folders are automatically compared again. If you do not need a second comparison, then the process can be interrupted by clicking on the button "Abort" or press the key ESC on keyboard. If the re-comparison was not interrupted, and we turned on all the display buttons, then we will see this window

Everything. The assigned task has been completed. All identical files are found and deleted in the folder "Archive".

Educational video on the topic

2.Search for duplicates.

Fundamental difference this method from the directory synchronization method is that TC ignores the names of the compared files. In fact, it compares each file with each, and shows us identical files no matter how they are called ! Such a search is very convenient when you do not know the folder structure or the names of the files being compared. In any case, after searching for duplicates, you will get an exact list of identical files.

Finding duplicates I will show on one practical task, finding duplicates of personal photos. Quite often you dump photos from your digital gadgets. Often the situation gets confused, something is reset many times, something is skipped. How to quickly delete files reset multiple times? Very simple!

Let's get started.

Suppose you always dump all your photos into a folder "PHOTO" on drive D. After all the resets, the folder looks something like this:

As you can see, some files are in folders named after the shooting date, some are reset to the root of the folder "_New" and "_New1"

To start searching for duplicates, open the folder in which we will search in any TC panel. In our case, this is the folder "PHOTO"

Next, press the key combination on the keyboard ALT+F7 or choose from the menu "Commands" - "Search for files"

A window opens standard search TC. string "Search files:" leave empty, then all files will be compared.

Then go to bookmark "Additionally" and tick the boxes "Search for duplicates:", "by size", "by content" and press "To start searching".

The search can take a VERY long time, don't be afraid of it, as there are a huge number of comparisons of a large amount of files. At the same time, the status bar shows the percentage of completion.

When the search ends, a search results window will open, in which we press the button "Files to Panel"

In the search window and in the panel window, identical files are collected in sections separated by dotted lines

Each section displays the file name and full path to the file. Names of IDENTICAL files can be completely different!
In this case, it can be seen that the same photograph was recorded THREE times, and twice under the same name ( IMG_4187.JPG) and the third time this photo was recorded under a completely different name ( IMG_4187_13.JPG).

Then it remains to select unnecessary identical files and delete them. This can be done manually by highlighting each file by pressing the key Ins. But it takes a long time and is not effective. There are better and faster ways.

So, our task is to remove duplicate files in folders "_New" and "_New1".
To do this, press on the additional keyboard, on the right a large key [+] . Usually this key in TC selects files by mask. The same operation can be done through the menu "Select" - "Select group"

This opens a window "Highlight duplicates", in which we go to the tab "Catalogs"

Initially on the tab "Catalogs" all directories found during the search are selected

Put a tick "Also select subdirectories (when selected manually or by template)" and press the button "Remove from everyone". In this case, the selection in the list is removed from all directories

Now it remains to select manually, by clicking the mouse, those directories in the list in which you want to delete duplicate files. This will select all subdirectories in the selected directory.

Everyone, press the button "OK" and again we get to the panel with the found files. Only now all the necessary duplicate files are already marked in red. Pay attention to the section of three identical files. It highlights two of the three, as needed.

Everything, it remains only to delete the selected files in standard ways TC. Can be pressed on the keyboard F8, you can key DEL, you can press the button "F8 Delete" on the screen. TC will ask you again about your desire to delete files,

press "Yes" and that's it, the files are deleted! Only non-deleted unique files remained in the panel.

The method described above is not complete instructions to find and remove duplicates. In the window "Highlight duplicates" there are much more options for selecting files. A full description would take a very long place.

Educational video on the topic

Let these examples serve as a starting point for you to start effective manipulations with your files.

You may also note

27 comments

Surely any of us have duplicate files accumulated on the disk over time. Files in "Downloads" that you have downloaded several times, the same photos and musical compositions lying in the bowels so that your hands simply do not reach them. You can get rid of all this manually, but they will work much faster for you special utilities looking for identical files.

A very popular "cleaner", which, probably, is installed on everyone. Yes, it not only searches for system junk and clears history and browser cookies, but also remove duplicate files.

Platforms: Windows, Mac.

Price: free, $24.95 for the extended version.

The program searches for files with the same or similar names, as well as with identical content. Works well with music and can find the same music files even if they have different tags. In addition, dupeGuru can compare images to find not only the same, but simply similar photos.

Developed for Mac and Linux. The Windows version is no longer supported by the developer, but it can be downloaded from the official site - it is fully functional.

Platforms: Windows, Mac, Linux.

An advanced file search application that, among other things, can remove duplicates. SearchMyFiles has flexible filters so you can customize search results just the way you want.

Platforms: Windows.

A popular Mac application that finds the same or similar files and shows you the difference between them. Copies in "Photos", in the iTunes library - nothing will pass by Gemini 2. The developers have announced a smart duplicate search mechanism that remembers which files you leave and which you decide to delete.

Platforms: Mac.

Although AllDup is free, it does a lot. Searches, deletes, copies and moves duplicate files, including the same audio recordings with different tags. Present flexible setting search. Using the built-in viewer, you can examine the files and choose what to delete.

Platforms: Windows.

Duplicate File Finder finds duplicate files quickly and efficiently. Provides an interesting opportunity to search for duplicates not only on the hard drive, but also in local network. Can work with images and music, comparing both tags and content. The preview function will help you figure out what to really delete and what to leave. Unfortunately, in free version quite a few options are not available.

Platforms: Windows.

Price: free, $29.95 for the extended version.

Universal file manager, which can do anything with your files. Including find duplicate files. You can enable the search for copies option on the tab with search options, in the same place where other attributes of the searched files are specified.

Platforms: Windows.

DupeGuru looks like the most attractive option. It's free, but it offers all the features you need to rid your drive of accumulated junk. The only disappointing fact is that the development of the Windows version has been stopped. Windows users For those who don't want to pay for commercial alternatives, AllDup is more suitable. And CCleaner and Total Commander are more versatile and common solutions that everyone probably already has installed.

A modern hard drive is created with an impressive amount of free space for storing information. However, this does not mean that there is no need to "take care" of the disk, freeing it from garbage, duplicated files.

The presence of duplicate files clogs the disk with unnecessary records of information. When it's time to increase free space"littered" disks, significant time will be spent on detecting duplicates - after all, the volume of your hard drive is very large. Therefore, finding and deleting duplicate files must be done on time. Consider a number of applications that can search for such duplicate files.

Total Commander

Let's consider the popular Total Commander as the first application. His functionality wide, it also provides a tool for finding duplicate files located in different directories on your hard drive. The operation we need is available in the general search module.

The search module is called using hot keys or by traditionally going to the main menu, where you will need to select the "File Search" function. A call via hotkeys is made by simultaneously pressing the Alt + F7 combination.

The search module offers whole line tabs of general and advanced functions. However, initially we need to select the drive or directory in which we want to conduct discovery. This is done on the general options tab. Then, go to the "Advanced" tab.

In the search module, open "Disks" and select the "D" drive.

Let's go to the "Advanced" tab.

At the bottom of the selected section is a function for finding duplicate files. We only need to configure this feature by checking the required checkboxes next to the desired options. For example, as shown in the image below. In our case, search actions occur on volume D.

Found the desired duplicates Total Commander has in a small window in which you can barely see.

By clicking the button "Files to the panel", we will display them in the main window. Now, you can freely delete them if you are sure that such an action will not cause the programs and the entire system to become unstable.

We get the result in the main window.

Finding duplicate files in CCleaner

Recently updated CCleaner also provides a function to find duplicate files. This application is a well-known and popular "junk cleaner" on the PC, so the presence of such an option in it is absolutely natural. The "Search for duplicates" function is easy to find on the "Service" tab of the program.

CCleaner detects duplicate files by several parameters: name, size, date, content. It also offers a number of useful options that allow us to skip or add a number of special files to our operation: hidden, system, read-only, zero size. To start the search module, just click the "Find" button.

After performing the operation, the user has the opportunity to select and delete the detected duplicates.

Size Explorer Pro

This application is powerful tool examining disk and attached volumes disk devices. One of its functions is to search for duplicate files.

In order for SizeExplorer to start detecting duplicates, it first needs to analyze the section where it needs to search.

The menu that opens will offer you a number of functions, among which is "Find Duplicates". Clicking this option will open a dialog with search module configuration options.

Let's define the detection functions.

Let's search for duplicate files according to the size-name-content scheme. It will take some time to detect, but after the end of the process in a separate window we will see complete list duplicates. At the same time, the program gave us many duplicates of incomplete similarity, according to a certain percentage of similarity.

Search results for identical files:

This is how you can select duplicates:

Let's use Ainvo Duplicate File Finder

Another program of interest is available on the Internet for detecting duplicate files. This is Ainvo Duplicate File Finder. This application is easy to use, so even a not very experienced user can use it. computer technology. To use the program, just click the mouse "Search for identical files".

The application immediately starts the search, which lasts quite a long time, so you have to be patient.

First, the disks are examined, then the application processes the found material, and only after that additional processing occurs, the result of which is the detection of duplicate files. To see the found duplicates, you will need to open the "Identical files" window.

As you can see, two sections are selected in the settings for the operation.

Master Ainvo has completed the operation - now we can open a window with links to duplicates.

As you can see, there are a lot of duplicates and the program immediately highlights files that are repeated, in its opinion.

There are 2137 duplicates selected that can be removed using the wizard. Also, it is possible to view a duplicate report.

The report opens in a web browser.

Summary

The hard drive needs maintenance from time to time. One of the preventive procedures is cleaning the volumes of garbage, a large proportion of which are duplicated files. To detect duplicate files located in different directories, you can use special programs. Among the most worthy applications that can detect duplicate files are: Total Commander, CCleaner, SizeExplorer Pro, Ainvo Duplicate File Finder.

Deleting duplicate files (copies of the same file) or folders can free up disk space and reduce unnecessary garbage in the system, which has a positive effect on performance systems. Sometimes duplicates are created by the user himself (for example, the same photos are copied to different folders located inside the same physical disk), in some cases they remain after using different software. The function of searching for unnecessary copies is performed by many applications, most of which are distributed under the free status ( is free). Let us describe the operation of several such applications.

Removing duplicates with Total Commander

Using Dupeguru

This software is designed solely to scan the system for copies and does not perform other functions.

AllDup to identify duplicates

The design of the program is made in a beautiful and easy-to-use style. Immediately after starting, a window appears with a brief leadership to find duplicates. You can search the entire physical disk by ticking the boxes you need local drives.

Or add separate folders by clicking on the arrow next to the " Source Folders» and selecting in the drop-down list « Add folders».

Let's try to find duplicates in the Games folder in the local drive "C:\"

Activate the tab Search Method”, set the criteria. By default, the option to search for duplicates only by similar names, if necessary, tick the necessary settings. It is advisable to enable search and extensions, otherwise the software may show files with similar names, even if they are not copies of each other.

After, press the button search start.

When completed, a new window will open with results, in which we mark junk files , click right click mouse by file, which is supposed to be cleared of duplicates (open context menu) and choose " Remove all other files from this group permanently to delete copies. In this case, the original itself will remain untouched, only duplicates will be deleted.

How to use DuplicateCleaner Free

A very thoughtful application for finding and removing duplicates on a computer. At the same time, it is possible to perform Search all files in the normal mode, look for copies of images or music, or identical folders.

Launching the application. First of all, we set up the search criteria, for this, in the section " Additional options » mark the setting « With the same name"and go to the tab" scan path»

Choose a folder, press the button additions the selected folder to the included paths and click the button " Scan».

After constant use of a computer, large amounts of data accumulate on its disks, i.e. all sorts of photos, videos, films, music, documents, and so on. This data takes up a lot of space. But very often duplicate files take up too much space.

In Windows, unfortunately, there are no normal built-in tools for finding duplicate files. There is an option to do this via command line PowerShell, but this is very inconvenient, especially for beginners it will be difficult. Therefore, it is easier to use third party programs. One of these is called AllDup. It is completely free, available in Russian, supported by all operating systems. Windows systems and finally quite easy to use.

Downloading and installing AllDup

The program can be downloaded for free from the AllDup official website. Below is the link to the download section:

The program is available in 2 versions: regular installation and portable (Portable). Portable is different in that it does not require installation on a computer, i.e. the program can be run directly from the downloaded folder.

To download, press the button "Server # 1", or "Server # 2" or "Server # 3" (if the first button does not download, spare servers are given) under desired version programs.

For fresh versions, always refer to the official website of AllDup!

Installing the program is very simple, it can be said that it consists of successive clicks "Next", no special settings do not need to be produced. So this process will not be considered. See also How to disable or remove OneDrive in Windows 10 and.

Finding duplicates with AllDup

After you have installed the program, run it. The main window for search settings will open:

Search setup includes several steps:

Selecting folders on your drives where AllDup will look for duplicates.

Folders are selected in the Source Folders section. You can mark both entire local drives and individual folders on your computer. Usually, users roughly guess where duplicates can be located, and in this case it is better to select specific folders, because searching the entire disks, firstly, can take a lot of time (depending on the amount of stored data), and secondly, it will give you a bunch of duplicates of those files that you are not interested in (for example, duplicates of any system Windows files which are best left untouched).

So, select the necessary folders / drives to search in the window by checking the boxes in front of them. If the desired folder is not in the list, you can add it manually by right-clicking (hereinafter "RMB") in the window and selecting "Add Folder".



Explorer will open, where you just have to select the desired folder and it will be added to the list.

It is best to leave the rest of the parameters as they are at this stage. See also Ways to Crack Your Passwords and Time Limits in Windows 10.

Method for finding duplicates.

This is an important step where you configure what criteria AllDup will use to determine if there are duplicates. You can set multiple criteria at once. The thing is that not always similar files can be considered duplicates. For example, you have 2 identical photos, but one of them has a small black dot, even if it is 1 pixel in size. In this case, it's a different photo because its content is different. On the other hand, these 2 photos may have the same file name, the same file extension, and if you select the last 3 criteria, the program will consider these photos as the same when searching.

• File name;

  File extension;

  File size;

  File contents.

With this choice, you will surely find the true duplicate files without error.

Thus, mark the criteria and nothing else can be configured at this stage.



File filtering.

You can configure the program so that during the search it finds only files with the extensions you need, for example, only JPG photos, only MP4 videos. This is very convenient when you know what duplicates you want to get rid of first of all (for example, from a video, because it can take up a lot of space) and then you don’t want to search through all the files in a row in the search results, of which there can be thousands. Or you can filter files by size, for example, find only those whose size is more than 1 GB.

Thus, on the "File filter" tab, enable the "Use file filter" option (1), check "Inclusive" (2), and then below (3) select the file extensions that the program will find. Files of any other extensions will not be displayed in the search results.



Most likely, the list of extensions will not contain all that you may need. To add an extension there, right-click and select "Add filter by entering text."



In the window that opens (1), enter *.file_extension, where instead of "file_extension", respectively, specify the file extension you need so that the program finds these files in the search.



For example, if you want the program to find video in AVI format, write *.avi and click "OK" (2). The symbol "*" means that the names of the files can be anything.

At the same stage, you can set up filtering of files as a result of the search by their size, for example, to filter out those that are smaller than the size you specified. Specify the desired options at the bottom, under the heading "File Size". In my example, an option is enabled that excludes files that are less than 500 KB in size. You will also be interested in the User Action Recorder in case of problems and The computer and laptop on Windows 10 do not start. How to fix it? .

These are all the main stages of setting up a search, the rest can be omitted.

Now, to start searching for duplicates, click the "Search" button at the top of the AllDup window:

The search process will begin.

How more files in the folders you specified is located on your drives, the longer the search will take.

After the search is completed, the program will display the found files with duplicates in the form of a table.

The first thing to do right away is to save the search results, because if you close this window with the results now, then the search will have to be performed again. To save, click the button with the image of a floppy disk, or select "Search result" in the top menu and click "Save search result".

Now, even if you turn off the computer, after starting the program again, you will be able to get to the search results again.

You can sort the search results by different parameters by clicking on the headings by a column in the table. The most useful, in my opinion, criterion for sorting is the size of the files. Therefore, if you want the largest files found to be displayed at the top of the table, then click on the "Size (Bytes)" column.

The next thing to adjust for the convenience of viewing the results is the displayed size. Initially, the program shows the file size in bytes, which is not very convenient. It is better to set the display in megabytes or even gigabytes. To do this, click the button marked in the screenshot below (1), then check one of the options (2):

Now I will dwell on how, in fact, to use the search results, how to view and remove unnecessary duplicates ...

The program divides the found duplicates into so-called groups. One group is all found copies of the same file, including the original (it will also be displayed in this group).

To view duplicates of one of the groups, you need to open it by clicking on the arrow. Example:

After expanding a certain group, you can check what kind of file it is by opening it. To do this, simply double-click on the file in the group, or right-click and select "Open File". The file will be opened in standard program Windows, through which you usually open all files of the selected type.

To delete duplicates, mark them with a checkmark, right-click and select one of the options: delete the file in the Windows recycle bin, or permanently delete it.

Accordingly, do not delete all files from the group, because this way you will delete both the duplicates and the original at once! For example, if there are 3 files in a group, then deleting 3 at once will delete both the original and 2 duplicates. In this case, to keep only one single copy of the file, you need to remove 2 files from the group.

Thus, you can check each group separately and remove duplicates. But if a lot of information is found, you can make it easier. Make it so that the program automatically selects all files in each group, except for one (i.e., only duplicates), after which you can get rid of all duplicates at once or run through and double-check whether everything marked is to be deleted.

To automatically mark duplicates, go to the "Select" menu (1) and check one of the options (2), for example, "Select all files except the first file".

As a result, the program will select 2 duplicates in each group, and leave the first file in the list unselected. Those. this way you will mark 2 duplicates and the original will remain unmarked. Or you can use the "Select" menu to try other options that are convenient for you.

After the program has marked the files, you can double-check the selection if necessary. And to quickly delete everything unnecessary or perform some other action, click the button marked in the screenshot below:

In the window that opens, you will see the total volume of the selected files, i.e. how much space the found duplicates occupy and the number of selected files. At the bottom, you need to select an action on the selected files. You can delete files through the recycle bin, delete them permanently (delete files item), copy or move files to any folder, and also rename found duplicates. If you are sure that the marked files are duplicates and you no longer need them, then it is easier to delete them, but in any case, the choice is yours.

So, select the desired action (1) and click "OK" (2). There is nothing more to configure here.

After that, the program will perform the action you have chosen on the previously marked files!

That's the whole process in general :) To exit the search results, just close this window. If you saved the search results, then, if you need this result again, you can get to it through the "Search result" section (1) in the main program window. The results you saved will be displayed in the table (2). To open the desired result, simply double-click on it.

Conclusion

AllDup - very handy program to find duplicate files on your computer. There is essentially nothing superfluous in the program, it has everything the right tools, filters and parameters for fast processing of a large volume of found duplicates. Of course, there are similar programs that, for sure, also do their job well. So far I've tried only AllDup and I don't see any point in changing it yet.