Limits...
Using workflows to explore and optimise named entity recognition for chemistry.

Kolluru B, Hawizy L, Murray-Rust P, Tsujii J, Ananiadou S - PLoS ONE (2011)

Bottom Line: Results indicate that, for chemistry in particular, eliminating noise generated by tokenisation techniques lead to a slightly better performance than others, in terms of named entity recognition (NER) accuracy.On the Sciborg corpus, the workflow based system, which uses a new tokeniser whilst retaining the same MEMM component, increases the F-score from 82.35% to 84.44%.On the PubMed corpus, it recorded an F-score of 84.84% as against 84.23% by OSCAR.

View Article: PubMed Central - PubMed

Affiliation: National Centre for Text Mining, Manchester Interdisciplinary Biocentre, University of Manchester, Manchester, United Kingdom. balakrishna.kolluru@manchester.ac.uk

ABSTRACT
Chemistry text mining tools should be interoperable and adaptable regardless of system-level implementation, installation or even programming issues. We aim to abstract the functionality of these tools from the underlying implementation via reconfigurable workflows for automatically identifying chemical names. To achieve this, we refactored an established named entity recogniser (in the chemistry domain), OSCAR and studied the impact of each component on the net performance. We developed two reconfigurable workflows from OSCAR using an interoperable text mining framework, U-Compare. These workflows can be altered using the drag-&-drop mechanism of the graphical user interface of U-Compare. These workflows also provide a platform to study the relationship between text mining components such as tokenisation and named entity recognition (using maximum entropy Markov model (MEMM) and pattern recognition based classifiers). Results indicate that, for chemistry in particular, eliminating noise generated by tokenisation techniques lead to a slightly better performance than others, in terms of named entity recognition (NER) accuracy. Poor tokenisation translates into poorer input to the classifier components which in turn leads to an increase in Type I or Type II errors, thus, lowering the overall performance. On the Sciborg corpus, the workflow based system, which uses a new tokeniser whilst retaining the same MEMM component, increases the F-score from 82.35% to 84.44%. On the PubMed corpus, it recorded an F-score of 84.84% as against 84.23% by OSCAR.

Show MeSH

Related in: MedlinePlus

U-Compare output for a test document.Chemical names (underlined) as identified by the MEMM-based workflow.
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC3102085&req=5

pone-0020181-g006: U-Compare output for a test document.Chemical names (underlined) as identified by the MEMM-based workflow.

Mentions: Figure 6 shows the chemical names as annotated by the Oscar workflow in the U-Compare framework. When these entities (which are underlined) are clicked, more information about the entity such as confidence scores, metadata etc. is available to the user.


Using workflows to explore and optimise named entity recognition for chemistry.

Kolluru B, Hawizy L, Murray-Rust P, Tsujii J, Ananiadou S - PLoS ONE (2011)

U-Compare output for a test document.Chemical names (underlined) as identified by the MEMM-based workflow.
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC3102085&req=5

pone-0020181-g006: U-Compare output for a test document.Chemical names (underlined) as identified by the MEMM-based workflow.
Mentions: Figure 6 shows the chemical names as annotated by the Oscar workflow in the U-Compare framework. When these entities (which are underlined) are clicked, more information about the entity such as confidence scores, metadata etc. is available to the user.

Bottom Line: Results indicate that, for chemistry in particular, eliminating noise generated by tokenisation techniques lead to a slightly better performance than others, in terms of named entity recognition (NER) accuracy.On the Sciborg corpus, the workflow based system, which uses a new tokeniser whilst retaining the same MEMM component, increases the F-score from 82.35% to 84.44%.On the PubMed corpus, it recorded an F-score of 84.84% as against 84.23% by OSCAR.

View Article: PubMed Central - PubMed

Affiliation: National Centre for Text Mining, Manchester Interdisciplinary Biocentre, University of Manchester, Manchester, United Kingdom. balakrishna.kolluru@manchester.ac.uk

ABSTRACT
Chemistry text mining tools should be interoperable and adaptable regardless of system-level implementation, installation or even programming issues. We aim to abstract the functionality of these tools from the underlying implementation via reconfigurable workflows for automatically identifying chemical names. To achieve this, we refactored an established named entity recogniser (in the chemistry domain), OSCAR and studied the impact of each component on the net performance. We developed two reconfigurable workflows from OSCAR using an interoperable text mining framework, U-Compare. These workflows can be altered using the drag-&-drop mechanism of the graphical user interface of U-Compare. These workflows also provide a platform to study the relationship between text mining components such as tokenisation and named entity recognition (using maximum entropy Markov model (MEMM) and pattern recognition based classifiers). Results indicate that, for chemistry in particular, eliminating noise generated by tokenisation techniques lead to a slightly better performance than others, in terms of named entity recognition (NER) accuracy. Poor tokenisation translates into poorer input to the classifier components which in turn leads to an increase in Type I or Type II errors, thus, lowering the overall performance. On the Sciborg corpus, the workflow based system, which uses a new tokeniser whilst retaining the same MEMM component, increases the F-score from 82.35% to 84.44%. On the PubMed corpus, it recorded an F-score of 84.84% as against 84.23% by OSCAR.

Show MeSH
Related in: MedlinePlus