1. PolySearch2 Evaluations
2. PolySearch2 Evaluations using BioASQ dataset

To assess the performance of PolySearch2, we conducted a speed test comparing only the speed of the original PolySearch with PolySearch2 on various queries with equivalent parameters. We then performed four evaluations to compare their accuracy. Finally three additional evaluations were conducted to assess the performance of PolySearch2 on several novel search tasks. Performance statistics including precision, recall, f-measure, and accuracy are presented in Table 1 for the 7 evaluations. Table 1 also lists the feature differences between PolySearch and PolySearch2.

Table 1 summarizes the performance evaluation and feature comparison of PolySearch 2.0 versus the original PolySearch. Evaluation #1 assesses PolySearch2’s ability to identify disease-gene association. Evaluation #2 assesses PolySearch2’s ability to identify drug-gene/protein associations. Evaluation #3 assesses PolySearch2’s ability to identify protein-protein interactions. Evaluation #4 assesses PolySearch2’s metabolite-gene associations. Evaluation #5 assesses PolySearch2’s ability to identify drugs with significant adverse effects, or ‘dangerous drugs’. Evaluation #6 assesses PolySearch2’s ability to identify toxin-disease association. Finally Evaluation #7 evaluates PolySearch2’s ability to identify toxin-adverse effect associations. Analysis speed is calculated based on multiple runs on query with 10,000 relevant documents.

All Evaluation datasets are available at the Downloads Page.

To assess the flexibility of PolySearch2, we conducted an association test using BioASQ, a biomedical semantic Question Answering challenge's gold standard training dataset (Task 3B Training Set, released March 2015), and assessed PolySearch2's performance in finding associated disease concepts when presented with free-text sentences.

Table 2: Performance evaluation using the BioASQ Task 3B (biomedical semantic QA) gold standard training dataset. The search queries are question sententences from BioASQ and PolySearch2's disease association results are compared with tagged disease concepts in the BioASQ 3B gold standard training data set.