Mod Deduplicator

[yuca]

0. Download
1. What is it?        
2. Pros & Cons        - (why) should I use it
3. Requirements
4. How does it work
5. Introduction    - Fast start
6. In Detail
7. FAQ
A. Bugs
B. To Dos
C. Versions - Change Log
D. Legal

0. Download

Please read topics 1 to 5 once before using
WWW Download v1.10
Mediafire Download v1.10



1. What is it?    

GTRW2 is a file de duplicator combined with a mod manager. Its eats folders with files, stores unique files in an defined area and keeps track what was in the folder originally included and
sorts it in a category. On demand you can search for the folder (by category or name) and extract it to get the original back.

A quick example, you have 2 folders with the same mod but in different version. The app will store only unique files and the name of the folders.
You will save the amount of the duplicate files and the hassle to sort it.
If you by mistake add again the same mod, the app will not add any new files, just adds an entry.
    
This is an WiP APP, it works, and will have an upward compatibility.

Be warned, this tool is powerful, can lead to data loose. After every action, please check the status window for errors.
    
    
2. Pros & Cons        - (why) should I use it

This tool can help you, if you are working with many duplicating files (and have no time to sort) or you running out of space
This tool can help you, if you lost with your files and are unable to find something
This tool can help you, if you want to have make snapshots of your game (without been penalised for hard disc usage)

You should not use this tool, if your packages/mods consist of big single files (like rF2 mods or EXE) or seldom duplicating

You can use it for everything, what is not daily needed and needs somehow sorting


Pros

sort your data/mods
place independent, you can work with relative paths and move so your library to another drive/computer
individual categories
store where & what you want
no install

Cons

it anonymizes the files (you can't recover the data if the database is lost)
it stores the data unpacked (use folder compression to overcome this mostly)
uses a database


3. Requirements

Right now, it needs an running mySQL server to store the data
It was tested on an Win64 environment, but should work on any Windows
It needs an Graphic Adapter supporting OpenGL


4. How does it work

After setting up the App, you show it to a place where you stored you decompressed Mods, each in one folder.
It reads the files in the folder and compares if the file is already in the STORAGE.
If no, it will copied inside and indexed.
If yes, the MOD will use the former file stored when extracted.
After worked through the files, the gathered info is stored in the database under the name of the folder.
On extracting, the DATABASE looks which files are included in the Mod.
There are copied to the extracting path inside a folder called like the mod name.



5. Introduction    - Fast start

5.1. Preparation
Decompress the package where you want (if you want to store the data relative, check if the drive has enough free space).
Edit your categories in the types.txt file. Each line is the name of a category (no empty line at the end).
You can change the categories-names afterwards and add more. Categories could be different games or mod types or locations or anything.
Start your mySQL.
Start the App (you can create a link to the desktop, but the home directory needs to be setup).

Generate an Database Entry


press DATABASE

press CREATE NEW mySQL DB

Edit settings for your mySQL DB.
If you use XAMPP you just need to check the store path where your files should be copied, but you can use store/mods for your first database


press CREATE DB

If no errors (see STATUS window) we can go on activate the created DB



press CHANGE DB SOURCE


select database



5.2. Add some Packages


Prepare a folder with packages you want to store, remember the folder names will be taken as package names, so name them sensible (like "GTR2 v1.1 STEAM 2013.09.11 Snapshot")


press ADD


Edit the directory path to the path where your packages are.
The App will scan the path immediately after typing and show all directories.


choose the category the packages should be stored as


press ADD MODS
After some time the result is showed


5.3. Extraction


press MODS


press GRAB

a list of first 30 entries will show up (depending on your filter)


select desired package


change destination path


press EXTRACT


done extracting





6. In Detail

6.1. Setting Up

6.1.1. mySQL
If you do not use mySQL until now and have no idea how to set it up, there is an package called XAMPP on the internet (http://www.apachefriends.org/en/xampp-windows.html)
It is a full Apache/PHP/Perl/mySQL package preconfigured. Just install it and start mySQL with the XAMPP control panel and that’s it.

6.1.2. The category file
Until now, there is one category file for all databases. Each line is the name of a category.
If you need different categories for different databases, then you need another instantation of this app.
If you use non ASCII chars, this file needs to be saved as UTF8


6.2. Adding MODS
6.3. Mod handling
6.4. Extracting
6.5. Database

6.5.1

6.5.2

6.5.3 STORAGE
When inserting a path for store the files, you can use a relative path (i.e. store/mods), then the files will be stored inside the app folder.
If you want to store the data on a certain position, you can do so by writing the complete path (i.e. D:\MY MODS ).
You can always move the data, but you need to change the database settings to show where the data is stored.

You should not mix two database storages. The result is fatal.

For performance reasons, you should use another drive for the STORAGE as the source or extraction path. Remember, the files will be copied.

6.6. Settings

7. FAQ

7.1. How to change predefined path (in the add path)? Check the Language Text File. Some of the defined textes are there. Change it and save.
7.2. Can i start the App twice? Yes, but adding and/or deleting on the same database can lead to unpredicted results.

A. Bugs

A.1. The GUI does not allow usage of the ALT GR key. If you need a char which can be created only by this key, then CUT&PASTE it. After pressing ALTGR you need to refocus the Box to be able to write something.
A.2. The Window can not be resized
A.3. ************ fixed in v0.403 During processing, there is no refresh
A.4. ************ fixed in v0.410 Date in log shown is month old
A.5. ************ fixed in v0.414 Big file sizes shows as negative

B. To Dos

B.1. SQLite as alternative for mySQL
B.2. mySQL Database Backup
B.3. Mod Duplicate Console (to find / remove double entries)
B.4. Disable Buttons during long operations

C. Versions - Change Log

v1.101 - 2013.10.20
GUI issues
double starts
changed packet size to MB

v0.420 - 2013.09.24
mysql problem solved (filenames with ')

v0.414 - 2013.09.17
month fix
md5 fix for files with obscure filename

v0.411 - 2013.09.16
month fix
md5 fix for files with obscure filename

v0.407 - 2013.09.13
changed mod info title row
added % values in add mod window

v0.403 - 2013.09.13
refresh during long operations

v0.401 - 2013.09.13
addapted new GUI version

v0.391 - 2013.09.12
added small overwrite check (mod extraction)
added row info in mod file listbox
added extraction comment in mod listbox (after extraction)

v0.386 - 2013.09.11
first public release

D. Legal

By using, copying or distributing my software and accompanying documentation (the program's object code and documentation are collectively referred to as „the software”), you indicate your acceptance of the following license terms:

Usage
I grant you (the licensee) a permission to use the software at no cost, for non-commercial purposes on any computer or device in your possession. You may also make copies and distribute the software according to requirements of section below.

Warranties and Disclaimer
The software and any support from me are provided “AS IS” and without warranty, express or implied. I specifically disclaim any implied warranties of merchantability and fitness for a particular purpose. In no event will i be liable for any damages, including but not limited to any lost profits, lost savings or any incidental or consequential damages, whether resulting from impaired or lost data, software or computer failure or any other cause, or for any other claim by the user or for any third party claim.

Limit of Liability
Subject to any liability which may not be excluded or limited by law the licensor shall not be liable and hereby expressly excludes all liability for loss or damage howsoever and whenever caused to you.

Distribution
As used in this agreement, the term “distribute” (and its variants) includes making the Software available (either intentionally or unintentionally) to third parties for copying or use.You may not impose any further restrictions on the recipients' exercise of the rights granted herein.

You may distribute the software under the following conditions:
You may not modify, combine other commercial applications with, or otherwise prepare derivative works of the software. You may not distribute the software for a fee (by payment of money or otherwise, whether direct or indirect). In particular (but not limited to), you may not charge for the software to recover media costs associated with its distribution, you may not charge for a product or compilation the software is a part of, you may not provide a paid download of the software and you may not require a registration of some kind (for example, on your website) for downloading the software. you may personally install the software as part of a free or paid service. you may distribute the software along with computer hardware, as long as there is a reference to the inclusion of the software (for example, a printed logo or a desktop icon on preinstalled computer systems). You must not distribute the software in a manner, which arouses the impression that the software cannot be obtained for free or involves fees for its usage.

 

Edited October 29, 2013 by yuca