Check code_entity()

[henriquesposito]

Entity recognition with NLP in R used in manypkgs seem to not be working very well...
Since we have a PR open there, we can re-evaluate its' usage and/or alternatives.

[henriquesposito]

New 'nametagger' package available on CRAN https://cran.r-project.org/web/packages/nametagger/nametagger.pdf

[henriquesposito]

It seems that the NLP pacakges in R have been updated recently to include newest versions of the NLP models. This might make entity recognition more accurate but would require we setup our own wrapper functions to use it (instead of relying in the entity package from GitHub).

Another option is using the spacy python library in R with the spacyr package. It is one of the most accurate and developed libraries, though, using spacyr requires several setup steps related to python and miniconda that can be very painful (especially if you have a mac with an M1 chip and above as I do...)

[henriquesposito]

I have tried with spacyr and with NLP, the results are very similar ... Good, but not great.

I am assuming, if we want to keep on pursuing the NER route, we might need create custom annotators (to get the specific entities we want from titles), hand-code some observations, and setup this as a training data to code entities we want more accurately. @jaeltan what do you think? Should we undertake this work (i.e. how important is entity for us)? Alternatively, we can go back to use word matching to code such entities as we do with types or domain?

Regardless of the route we choose, what are some entities we want to extract?

• Rivers and oceans?
• IOs or NGOs?
• Biomes?

[jaeltan]

I think we would be interested in identifying actors like IGOs, NGOs, and private actors like companies in future, but I'm not sure this will be done only from titles of agreements, so maybe further expanding code_entity() is not the best route to pursue this and we can leave it as it is for now?