Crossover: 2014

Chapter 82 Algorithms that lead a generation and a half

And what is Lin Hui doing in the whirlpool of public opinion at this time?

Of course, I was looking up those "reference materials" that spanned time and space.

There is a lot of valuable information here.

Many things that are commonplace in later generations are indiscriminate killings in this time and space.

But Lin Hui was not swayed by the excitement.

Because Lin Hui always knows that half a step ahead is a pioneer, and one step ahead is a martyr.

It's okay to say that the technology is really ahead of others by a year or so, but if it is suddenly four or five years ahead of others, there will be big problems.

Only technologies that can fit the background of the 14 era are the valuable things Lin Hui is looking for.

In the ThinkPad in the previous life, I hardly searched for a long time.

Lin Hui found his prey:

——Algorithm for generating/extracting composite news summaries.

This algorithm was not particularly new in the previous life.

Lin Hui took a fancy to it because the algorithm was mature.

A certain degree of maturity means stability and reliable performance.

Lin Hui used this algorithm to develop the news summary software he had conceived without any additional training.

Of course, this algorithm is only not new in the time and space of the previous life.

In the time and space of 14, it is still a leading technology.

Just a little bit ahead, does it work?

Don't say that we are ahead by a little bit, but by half a point.

It still makes you desperate!
Before you broke my monopoly, I made huge profits by blackmailing at high prices.

You have broken my monopoly, and I will directly follow you in the price of cabbage.

Are you angry?
I don't know if others are angry or not.

Anyway, the rabbit was already so angry that he wanted to bite.

What's more, the iteration of algorithms is inherently fast!
A year and a half ahead is almost equivalent to being a generation ahead in terms of technology.

The generation/extraction composite news summary algorithm is one and a half generations ahead of the mainstream news summary algorithm in time and space for 14 years.

This is not an exaggeration by Lin Hui.

In fact, the current way of automatically generating news summaries is still extractive news summaries.

Extraction, as the name suggests, is to find one or several sentences closest to the central idea from the original news text according to a certain weight.

Extractive summarization is still using the "old" Text Rank sorting algorithm.

The general idea of ​​this algorithm is to remove some stop words in the article first.

Afterwards, the similarity of sentences is measured, and the similarity score of each sentence relative to another sentence is calculated.

Propagate iteratively until the error is less than 0.0001.

Then sort the key sentences obtained above in order to obtain the desired summary.

Objectively speaking, this algorithm is not bad.

But the problem is that extractive summarization mainly considers word frequency, and does not consider too much semantic information.

Because of this, it is difficult for this extractive summary to obtain the core content of complex news.

And an extremely obvious drawback of this summary method is:

Extractive summaries work fairly well for English news.

But for Chinese news, I am completely at a loss.

All in all, although the extractive summary is relatively mature at present.

However, the extraction quality and content fluency are not enough.

Because of the various shortcomings of extractive summarization.

Then came the generative summarization algorithm.

Generative summarization algorithms have benefited from the in-depth study of neural network learning.

This summarization generates summaries in a more human-like way.

This requires the generative model to have a stronger ability to represent, understand, and generate text.

The generative formula is that after the computer reads through the original text, it generates a smooth summary according to the machine's own words on the basis of understanding the meaning of the entire article.

Generative news summarization mainly relies on the deep neural network structure.

Generative summarization has inherent advantages over extractive summarization in understanding news content.

But this summary is not entirely without its drawbacks.

This way of summarizing is easily constrained by the length of the original text.

When putting a long piece of news in front of a generative summarization algorithm.

The probability of its performance is: (⊙﹏⊙) It's too long to watch!

The generating/extracting composite news summarization algorithm combines the advantages of the extractive summarization algorithm and the generative summarization algorithm.

For longer news, the algorithm can be used to extract the core content first.

Then generate based on the core content.

All in all, if a software is developed based on the algorithm for generating/extracting compound news summaries

It is also perfectly capable of beating the software developed by Nick D'Aloisio.

After all Nick develops the software.

Either Summly or Yahoo News Digest (Yahoo News Digest)
These are all based on extractive algorithms.

The generation/extraction compound news summary algorithm can be said to be a sling extractive summary algorithm in terms of efficiency.

But having said that, such a hanging algorithm only develops a software and then sells it.

It seems to be a bit of a loss.

How to say is also ahead of the times technology.

It seems that you can write a few papers or something.

Uh, but publishing a thesis right out of high school seems a bit shocking.

How to do to get the most out of it?

(End of this chapter)