Machine Learning Developer in London, United Kingdom

Relation Extraction (RE) aims to label relations between groups of marked entities in raw text. Most current RE models learn context-aware representations of the target entities used to establish a relation between them. This works well for intra-sentence RE and we call them first-order relations. However, this methodology can sometimes fail to capture complex and long dependencies. To address this, we hypothesize that at times, two target entities can be explicitly connected via a context token. We refer to such indirect relations as second-order relations and describe an efficient implementation for computing them. These second-order relation scores are then combined with first-order relation scores. Our empirical results show that the proposed method leads to state-of-the-art performance over two biomedical datasets.

Biomedical text tagging systems are plagued by the dearth of labeled training data. There have been recent attempts at using pre-trained encoders to deal with this issue. A pre-trained encoder provides a representation of the input text, which is then fed to task-specific layers for classification. The entire network is fine-tuned on the labeled data from the target task. Unfortunately, a low-resource biomedical task often has too few labeled instances for satisfactory fine-tuning. Also, if the label space is large, it contains few or no labeled instances for the majority of labels. Most biomedical tagging systems treat labels as indexes, ignoring the fact that these labels are often concepts expressed in natural language, e.g., `Appearance of a lesion on brain imaging.' To address these issues, we proposed constructing extra labeled instances using label-text (i.e., label's name) as input for the corresponding label-index (i.e., label's index). In fact, we proposed a number of strategies for manufacturing multiple artificial labeled instances from a single label.

We proposed a model for tagging unstructured texts with an arbitrary number of terms drawn from a tree-structured vocabulary (i.e., an ontology). We treated this as a special case of sequence-to-sequence learning. The decoder begins at the root node of an ontological tree and recursively elects to expand child nodes as a function of the input text, the current node, and the latent decoder state. In our experiments, the proposed method outperformed state-of-the-art approaches on the important task of automatically assigning MeSH terms to biomedical abstracts.