Text mining on COVID19 datasets - Terminology extraction
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 45 | |
| Author | ||
| License | CC Attribution 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/53410 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Producer | ||
| Production Year | 2021 | |
| Production Place | Wageningen |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
MOOD Science Webinars5 / 45
9
13
24
25
26
35
36
38
40
42
43
44
00:00
Computer animation
08:57
Computer animation
17:54
Meeting/Interview
18:47
Meeting/InterviewComputer animation
19:19
Meeting/InterviewComputer animation
20:32
Meeting/Interview
22:18
Meeting/Interview
23:18
Meeting/Interview
Transcript: English(auto-generated)
00:08
My talk is about text mining and COVID-19 data set, but it's larger than my presentation, my purpose. I would like to focus on terminology extraction
00:22
and how we can take into account terminology extraction in the context of mood, and in the context of event-based surveillance systems. It's based on data text mining,
00:42
on informal sources like press, social media, and blogs. And the main challenge I would like to discuss, and at the end it would be nice to have this discussion, is how to use terminology and text mining for event-based surveillance systems. I would like to summarize and to sum up the different systems
01:03
we developed at CIRAD and in RIA, but called PADIweb. You know this system, but I think it could be interesting to summarize this pipeline.
01:20
So in output we have different text-to-read data, news data, and the objective of PADIweb is to extract epidemiology event in this text-to-read data, that this is host, location, death, number of cases, symptoms.
01:40
This software is dedicated to animal disease surveillance systems, and it's important to keep in mind this aspect and how we can have a generic approach in the 1L context.
02:01
So in the context of PADIweb, and in order to summarize the pipeline, the PADIweb pipeline, there are different steps. The first one is data collection, in order to collect news data like newspaper, for instance.
02:22
Currently we collect data from Google News. The second step is dedicated to data processing, in order to claim the text, to language detection, translation into English, because currently PADIweb is working on English data,
02:44
so the algorithms are working in English. The following step is dedicated to data classification using machine learning techniques, in order to identify automatically if a document is relevant or irrelevant
03:02
with epidemiology point of view. And at the end, we extract information into this text-to-read data, like your disease, host location, and so on. And in this context, terminology extraction
03:20
and the use of terminology is crucial. For all the different steps, and more precisely for data collection, the choice of the keywords in order to collect data is crucial for syndromic surveillance, for disease-based surveillance,
03:42
the name of the disease, the name of the symptoms, the different synonyms of the symptoms, and so on. It's crucial. Terminology extraction is crucial for classification. Actually, classification is based on a textual representation of text called bag of words,
04:05
and the choice of the words in order to predict if a document is relevant or irrelevant is very important for machine learning. And at the end, in order to extract information, the different keywords are important.
04:24
For instance, we can have different definitions of the symptoms with different variations of the definition of the symptoms, and it's important in an automatic way to consider this aspect.
04:41
So in the context of COVID-19, we investigated different terminology extraction strategies, and we have two different objectives. The first one is terminology extraction for surround systems and information extraction.
05:00
This is the first step and the last step of PADI-Wave, for example. And at the end, this is dedicated to the work conducted in the context of work packet three of mood, is a trend analysis of COVID-19 terminology per period and location. And it's the main subject of the paper
05:27
we can discuss today. In the context of this work, we used different kind of text-to-read data, and this is important to keep in mind this aspect
05:40
because the methods and the results are very different. The first kind of text-to-read data we take into consideration is a scientific publication dealing with COVID-19. Currently, we have a lot of publication available
06:03
in order to apply text mining methods, so this is important in all domains. The second one is media data, and more specifically, we would like to discuss the use of media data in this context,
06:22
and we use PADI-Wave data as well. And at the end, we use social media like tweet, and currently we investigate other kind of social media like YouTube data, and how we can consider this new social media
06:46
in the context of mood and surveillance systems in general. So the first objective is dedicated to terminology extraction. So this is just to explain the global pipeline
07:04
in order to extract terminology in the context of text mining. Currently, we use a software called Biotech implementing in the context of a PhD in Montpellier.
07:28
Currently, for terminology extraction in input of the process, we have a different corpora in this context, publications, scientific publication dealing with COVID-19,
07:40
and at the end, we have relevant terms extracted automatically in the text. There are different steps in order to extract relevant terms. The first one is a part of speech tagging in order to give a grammatical label for each word of the text
08:01
like adjective, noun, preposition, and so on, verb, and so on. The second step is to extract term according to syntactic patterns like noun, noun, non-proposition, noun, adjective, noun, and so on.
08:23
So there are generic patterns used in different languages, but we can use specific patterns dedicated to specific domains like biomedical domain, for instance. And at the end, we have a lot of terms, candidates,
08:42
and we would like to highlight and to rank the term using ranking measures based on statistical aspects and statistical measures. At the end of this slide, we have a measure with three different components. This is an example of measures used,
09:02
for instance, for the first component, the first component is dedicated to extract terms with relevant patterns for the domain in this context, biomedical domain. The second component is dedicated to extract discriminative terms.
09:21
I'm going to explain this aspect at the end of my talk. And the last component is to extract relevant multi-word term. I mean, in this example, we would like to follow the extraction of African swing fever and not African swing. This is a good example because in biomedical domain,
09:44
specific terms are more relevant and we use information about embedded terms in order to highlight a specific term like African swing fever in this example.
10:00
So, yeah, I have a problem, yes. And in the context of mood, we had a different strategy, different measure, different strategies in order to extract terms, for example, in publication,
10:23
we can extract terms in titles, in abstract, in the content of the publication, and we don't have exactly the same results. And the combination of this expression could be very attractive. We can use linguistic patterns in order to extract variation.
10:40
In this example, in this slide, we have different variation of terms. So this is very important for, for example, for syndromic surveillance using different variation of the term. And this is the objective of our work.
11:00
We evaluated this strategy, these different combination of methods for two tasks. So first one is COVID-19 surveillance and the second one is syndromic surveillance. For instance, it could be used for a known disease and we evaluated the relevant of the terms,
11:21
yeah, the relevant, yeah, yeah, thank you, the relevant of the variation extracted with all strategies. So the second steps, the second objective, the second objective for work conducted in the context of work package two
11:41
and work package three, is a trend analysis of COVID-19 terminology per period and location. And this is the main subject of the paper. We investigated the use of different corpora. The first one is media data, medicis data, I'm going to explain
12:01
and to summarize the use of these corpus. And the last one is social media data. So for medicis data, actually we collected data in 2020
12:22
for different periods, March, May and July in different countries, UK, Spain, France with different languages. And we adapted biotech in order to extract terms using different strategies dealing with the, this is an example,
12:42
with the word mask, the multi-word terms using the word mask. And this is a discussion at this period. It was a main discussion in the different countries at this period in order to compare the use of this vocabulary
13:02
according to the period and according to the country. So in order to finish, I would like to present a new work conducted currently in the context of mode.
13:23
Currently in text mining, we can use different strategies in order to extract discriminative terms, popular information which we will measure exists. And currently we adapted this measure in order to extract discriminative terms
13:41
per period and per countries automatically. So discriminative in text mining is based on the number of documents and in our context we use information, spatial information and temporal information
14:01
in order to extract discriminative term. The objective is to have, to monitor the vocabulary used in the different countries and during the different periods. This is the main objective of this work.
14:22
We implemented a new method. Now we are going to evaluate this method on different study cases. We would like to integrate sentiment analysis in this context in order to monitor the sentiment vocabulary used
14:40
and so on. This is the current work. So in order to conclude my short presentation, we have different challenges in the context of mood and surveillance system in general. The first one is about the generosity
15:01
of the approach. Currently PADIweb has been developed for animal disease surveillance and the main question is how we can have a one-horse approach in order to adapt PADIweb in the one-horse context for human disease but for plant disease too.
15:22
What is generic? What is specific? So this is an attractive and an important challenge today. The second challenge we have today is to extract information in social media and in noisy data like YouTube.
15:41
Currently we investigate the use of Twitter but we study the use of transcription of YouTube in order to extract information, spatial information, temporal information, thematic information according to the event, epidemiologic event, thematic information.
16:00
But these textual data are noisy. For instance, we don't have the punctuation and it's a natural language processing approach are not is given by the result in this context.
16:23
And the last aspect is about quality criteria. In the context of event-based surveillance systems we use informal resources and the question is how we can take into account the quality of the sources, the quality of the textual data,
16:41
the quality of the text mining method used in order to deal with this textual data. So it's important to have a global approach in order to take into account quality criteria in order to highlight relevant event in the context of this approach.
17:04
And the last aspect is how to connect, how to link the different kind of information we have. YouTube data, social media data, covariate data, other textual data, official information and so on.
17:22
So this is another challenge of work package 3 and work package 2 dedicated to text mining. I understand the work on text mining and that you can get how you can extract information and data
17:41
but how do you link that at the end to what we are looking at, outbreaks or events or sentiments and how do you see that in the I suppose the next step in the analysis? An important question. Currently we focus on the extraction of
18:02
information in order to define an event and we focus on the extraction of location, thematic information. And the key point is how we can define an event. Yesterday we had a discussion with Renaud
18:21
about the definition of which epidemiologic information we can extract in information. For instance, sentiment analysis could be used in order to extract a weak signal, how we can connect this weak signal with an event. For the connection with other kind of data like environmental data, official data,
18:43
I think we can connect with entity biomedical information and we have to have a normalization of the data point to keep in mind. I would like to understand better if the only way to search by using the terms
19:00
and the different synonyms is by listing them or there are alternatives that ensure a higher effectiveness of the search in order to don't miss any potential interesting topic or piece of information. Thank you. This is an important question.
19:21
We can have an input text and we can use different statistical measures in order to extract terms but we can adopt a driven extraction with a list of given terms in order to extract variation
19:42
using linguistic approach in this approach. And it's interesting to adopt this strategy because with variation we have the frequency of the term is low and with a statistical approach
20:00
we lost this information. So our terms are not extracted with statistical approach so using a linguistic approach with a list of terms and the extraction of variation with this approach
20:20
is crucial. We need information about terminology at different steps at the beginning of the process in order to collect relevant documents based on the choice of the keywords
20:42
and in this context the terminology is crucial for syndromic surveillance for disease-based surveillance with the name of the disease and so on. So at the beginning of the process the terminology is crucial. At the end as well
21:00
is important in order to extract event and epidemiologic information like symptoms and so on. We can have different variation of terms so an automatic extraction terminology extraction methods is important in this context.
21:21
With computer science point of view an important challenge is to mine data to apply text mining with noisy data with Twitter but with a transcription of YouTube it's an important question because we have noisy data
21:41
without punctuation and so on and text mining approach in general has a lot of problems. And another challenge is to extract and to identify weak signal in this kind of data and what is a weak signal. Another challenge is how we can consider
22:03
the quality of the data in order to highlight relevant information. The last one is how we can link all this data in the context of surveillance systems. The main challenge is how we can use
22:22
this kind of method in a one-house context with the genericity of the approach. We can have specific approach for specific disease but it's important to consider the genericity of the approach in the one-house context.
22:41
And I think another challenge is the multidisciplinary aspect how the expert could be integrated at the beginning of the system in order to collect data in order to identify relevant sources in order to adapt the different algorithms
23:02
with different parameters. So an expert has to know the behavior of the different algorithms and at the end in order to evaluate and to have a retrospective analysis of the result. This is a research project
23:21
so it's a combination of a new approach and a theoretical approach and a practical approach and in this kind of project like MUD it's important to consider new ranking measures new approach algorithms and application, real application we can use directly in the systems.
23:42
So we have to find a combination between both and I think this is very challenging because in the context of COVID-19 we have a lot of needs and it's important to keep in mind we need time in order to find new algorithms
24:02
in the research context.