Comparación de documentos dentro de MongoDB

I'm looking to compare fields between potentially millions of documents within a mongo collection. The fields will be determined ahead of time and weights will be given to each field. These weights will then be used to return document pairs representing suggestions for 'like' documents. For instance, if 2 documents are being compared and both have the same value for the field 'first_name', the weight table will be referenced and the score for the pair will have that weight added to it. If another field is the same between the two, the score will updated to reflect a higher likeness.

I'm currently approaching this by iterating through the initial result set, then having an embedded iteration that also goes through the result set and compares each document to the document that the first iterator is on (extremely inefficient). This is currently all done by php as it grabs elements through the cursor.

I'm open for any suggestions including MapReduce implementations (doesn't really seem applicable), cursor manipulation, pretty much whatever you can conjure up to simplify the process because im working at O(n^2) complexity right now (Well, a little better as I skip the documents that have been covered so far by the first iterator).

preguntado el 01 de febrero de 12 a las 22:02

1 Respuestas

To avoid n^2 you would have to look at storing fields and their values in a reference collection, e.g. :

{
    field: "firstName",
    value: "Remon",
    documents : [ <list with all document _ids of documents that have "field" set to "value">]
}

This way you can query directly on this collection to get all documents that are "like" your source document. Additionally this allows you to query for multiple key/value pairs with a single O(n) query.

Obviously the only tricky thing is maintaining this reference collection in the first place but in your case that seems pretty straightforward (update references when you update the fields).

¿Eso ayuda?

Respondido 02 Feb 12, 15:02

I looked at creating a setup pretty similar too this and was hoping to avoid creating additional collections to be maintained but this is looking more like the route I'm going to have to go. Thanks for the input. - Ghjnut

No es la respuesta que estás buscando? Examinar otras preguntas etiquetadas or haz tu propia pregunta.