-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathREADME
More file actions
37 lines (28 loc) · 1.19 KB
/
Copy pathREADME
File metadata and controls
37 lines (28 loc) · 1.19 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
Assumptions:
1. There is a folder called reuters21578 in the same directory as parser.py.
If not change line 7 of parser.py
Makefile stuff:
Might need to load module soft/gcc/4.9.2
Preprocess:
make preprocess
Build Executable (release):
make kmeans
Build Executable (debug):
make all
Clean slate:
make clean
To run:
./sphkmeans <input-file> <class-file> <#clusters> <#trials> <output-file>
Notes:
Initial centroids are randomly chosen document vectors.
Empty clusters are initialized by finding all the documents in the largest cluster and spliting it into two arbitrary halves.
If there is an empty cluster at the point of convergence, the above step is carried out and the algorithm tries again.
The only additional preprocessing I've done is remove all special HTML characters.
The list of such characters are found in spec_chars.py and begin with &.
The preprocessing takes about 5.5 mins to run.
All test cases should in total take less than 2 hours to run.
I usually run my code in the following manner:
python parser.py
make kmeans
python run.py
The last run.py runs ./sphkmeans for every possible combination of input-file and K and prints the output to the console.