Custom clustering code for making groups of related terms

Write custom clustering code for making groups if related terms. SOURCE: The source information will be a set of N-dimensional vectors, where N is a set of words that often appear in the same paragraphs as other words. The input are topics generated from a proprietary corpus using latent Dirichlet allocation (LDA). We currently have a dozen vectors (each vector is a topic from LDA), and N ~= 300. We have a simple file format delimitated with newlines, "|" and ";". OUTPUT: Code should be in a compiled language, such as Fortran or C. You will probably use Group Average Agglomerative Clusterer. We used python NLTK as a proof of concept, and we had preliminary success. You can see our simple python. There will be additional weighting information, as we have additional data about the weights of some of the other N words between eachother. The algorithm is intended to have the degree of clustering depend on the initial similarity of the clusters. There will be 5, tightly related tasks: 1) Write compiled code for merging our source vectors. The result will be analogous to our python NLTK sample. 2) Add weighting information we provide. (We have weighting scores for some of the N terms, which will cause any cluster they are in to be more or less important.). Specifically, we have 100 themes. Example themes are "sports" and "food". We know that the word "apple" has a high weight for the "food" theme, and a low score for the "sports" theme. Therefore a cluster containing [apple, THEME:sports] would be weighted lower than a cluster containing [apple, THEME:food]. 3) Adjust similarities for a subset M of N terms, so they are less likely to be combined. For example, if M = [orange, apple], then two sets [orange, banana] and [pear, apple] would be considered more distant. (not the subset M is the same as the THEMES in #2). Not all M have different relationships. Some are negative or positive. e.g., food:sports = -1; but computer:science = 0.8. We will provide a list. 4) Add information from an additional set of W vectors. These vectors are sets of terms extracted from Wikipedia. For example, a vector in W would be all the outgoing links from a wikipedia article, with higher weights depending on their closeness to the start of the wikipedia article. 5) Filter to omit stopwords (will be provided), irrelevant parts of speech (tbd), duplicates (i.e., no word should be in >1 final cluster), and low-probability groups (eliminated). The output will be a list of potentially related terms.