Crossover: 2014

Chapter 90: The Admiration of Top Algorithm Teams

Eve Carly is 25 years old and is a Ph.D. in computer science at the Massachusetts Institute of Technology.

In fact, the general computer majors seldom get a doctorate, and basically most of them go their separate ways after finishing their master's degree.

But Eve Carly has her own pursuit on the academic path.

Although this pursuit is almost doomed to go alone.

But she always enjoys it, and the biggest driving force along the way is interest.

The biggest reason besides interest is the pride brought by work.

She's also justifiably proud to be a member of the Text Summarization group at MIT's Natural Language Processing Research Program.

After all, the world's most efficient extractive text summarization algorithm was done by their team.

Eve Carly has always been proud of it.

However, this glory was gone half an hour ago.

A new text summarization algorithm stronger than the extractive text summarization algorithm developed by their team came out.

And it also appeared directly in Apple's App Store in the form of a mature application.

Eve Carly also learned about this after receiving Nick's help email.

In fact, she was a little skeptical when she just received the exaggerated help email from Nick.

She even thought at one point that Nick, the cocky, stupid lucky idiot, had misremembered the date of April Fools' Day.

The algorithm in the software that Nick uses is ostensibly the work of Esther's team.

But in fact the MIT Natural Language Processing Project Text Summarization group is the real source of the algorithm,

The algorithm used in Nick's software can be said to be the brainchild of everyone in Eve Carly's research group.

Eve Carly is still very confident in the algorithm that she personally participated in.

How can there be software algorithms that process news summaries more efficiently than the algorithms they developed?
It's not that she herself is arrogant and blindly confident.

In the past, the core algorithms of many software that appeared in the app store under the banner of news summaries were actually very inefficient.

Even many news digest programs that claim to be algorithmically unique have turned out to be nothing more than false claims.

The so-called Nanfeng APP claims that the surface efficiency is the strongest and the accuracy is the highest in the world.

At the beginning, Eve Carly just regarded these slogans as gimmicks, and didn't take them seriously.

However, the facts are quite embarrassing. Far from being a paper tiger, this Nanfeng APP can be called a peerless beast.

At least in terms of processing news summaries, the algorithm adopted by Nanfeng APP is extremely powerful in terms of efficiency.

After quantitative testing, Eve Carly found that the average speed of English news summaries in the 100-round test of Nanfeng APP was 241% faster than that of the software developed by Nick.

This is nothing, when the Nanfeng APP is run on a virtual machine with higher computing power.

The average speed of English news summaries in 100 rounds of tests is 350% faster than the average speed of summaries of their algorithm under the same conditions.

It can be said to be a full-scale sling.

Eve Carly couldn't understand how there could be an algorithm that was three times more efficient than the algorithm they developed in terms of extractive text summarization.

According to their research, the potential of current extractive text summarization algorithms has almost been exhausted.

Could it be that the algorithm team of Nanfeng APP has found a new way to squeeze the potential of the extractive text summarization algorithm?

Impossible, absolutely impossible.

No matter what, their research team is also a natural language processing algorithm team that brings together the world's leading technology giants.

It doesn't make sense that these elites will be caught up in the same direction.

If the Nanfeng APP algorithm team did not come from behind, it should be overtaking on a curve?
That is to say, the algorithm of Nanfeng APP is definitely not the traditional extractive text summarization algorithm, but a brand-new summarization algorithm.

The layman looks at the excitement and the insider looks at the doorway.

Eve Kali quickly verified her guess from the input and output results of several sets of news summary tests conducted by Nanfeng APP.

Nanfeng APP really uses a new text summarization algorithm.

As for the basis of judgment, it is very simple.

Extractive text summarization directly extracts words or complete phrases from the original text as the abstract of the article.

This process does not generate words and phrases that are not in the original news article.

However, Nanfeng APP will generate many words and phrases that are not in the original news text in the news summary.

That is to say, the algorithm used in Nanfeng APP is definitely not an extractive algorithm, at least not just an extractive algorithm.

And a major feature of this new algorithm in news summarization is that it will generate words and phrases that are not in the original news.

Compared with the traditional extractive text summarization, Eve Carly thinks this new summarization method in Nanfeng APP is more like a generative summarization method.

However, new questions immediately appeared in Eve Carly's mind.

How did the developer of this Nanfeng APP solve this brand-new algorithm tentatively called "generative summary algorithm"?
The so-called generative summarization algorithm and similar summarization algorithms based on neural networks have been dabbled in by their development team before.

At that time, they called this algorithm "Summary Summary Algorithm", but the actual performance of this algorithm was not ideal after multiple rounds of tests by their group.

Although this generalized or generative text summary algorithm can generate expressions that have not appeared in the original text, it is more flexible than the extractive summary algorithm.

But it is precisely because of this that generative summaries are more prone to factual errors. These errors include not only content that is contrary to the original text information, but also content that is contrary to people's common sense.

In addition, this kind of generative text summarization algorithm is easy to show obvious weakness when dealing with long news.

Although putting this generative summary algorithm and the extractive summary algorithm together will improve the ability of the generative summary algorithm to process news length.

However, after testing, there is no generative summary algorithm, and the extractive summary algorithm can perform more ideally.

To be on the safe side, Eve Carly's team finally chose the traditional text summarization direction by further strengthening the speed and accuracy of extractive text summarization.

A direction once abandoned by them, but picked up again by others?

It sounds a bit unbelievable, but the fact is that the developers of Nanfeng APP not only picked up the research direction they had abandoned, but did better than them, which can be said to be a severe slap in the face.

Eve Carly was a little confused. She couldn't figure out how the developers of the Nanfeng APP made a way in a direction they thought would not work.

But one thing is certain, although the developers of Nanfeng APP also use an algorithm similar to the generalization/generative algorithm, the specific generative algorithm itself is at least one generation more advanced than the generative algorithm they made in the first place.

Despite the confusion in her heart and the severe slap in the face, Eve Carly did not appear very emotional, at least not as emotional as Nick expressed in the letter.

Years of research career has long developed Eve Carly's rational character of not being surprised.

Furthermore, advances in technology have always been one after another.

If you worry about gains and losses because of temporary gains and losses, it is better to change careers as soon as possible.

Extra emotional fluctuations are not necessary, but will affect rational judgment.

After experiencing the Nanfeng APP in depth, Eve Carly had to admit that although this APP looks like a temporary translation software to make up the numbers, the core algorithm is indeed very strong.

Even as the slogan of this software says - "the strongest surface".

In addition, the summarization speed and summarization accuracy claimed by this software to crush similar software is also true.

Wait, remembering the "accuracy" emphasized in the promotional slogan of Nanfeng APP, Eve Carly suddenly thought of something.

The current news summary software algorithms all emphasize speed in terms of publicity, and rarely talk about accuracy in terms of accuracy.

It's not because accuracy is not important in news summaries, on the contrary, accuracy is extremely important in news summaries. It can be said that accuracy is the most fundamental factor to measure whether a summary algorithm is unusable, but there are few summary algorithms. An extremely precise quantification of accuracy.

There is no other reason, because the industry lacks a unified standard for measuring accuracy.

It sounds incredible, but the truth is, evaluating the accuracy of an abstract may seem easy, but it is actually a more difficult task.

For the measurement of an abstract, it is difficult to say that there is a standard answer. Unlike many tasks with objective evaluation criteria, the evaluation of an abstract depends on subjective judgment to a certain extent.

In summarization tasks, there is a lack of a unified scale for measuring the accuracy of summarization, such as grammatical correctness, language fluency, and completeness of key information.

There are two methods for evaluating the quality of automatic text summarization today: manual evaluation method and automatic evaluation method.

Manual evaluation is to invite several experts to manually formulate standards for manual evaluation. This method is closer to people's reading experience.

However, it is time-consuming and labor-intensive, not only cannot be used for the evaluation of large-scale automatic text summarization data, but also does not meet the application scenarios of automatic text summarization.

The most important thing is that if people with subjective thoughts evaluate the summary, it is easy to be biased. After all, there are a thousand Hamlets in the eyes of a thousand people. Everyone has their own criteria for measuring news summaries. Perhaps a measurement team A unified measurement standard can be developed, but it is likely that the measurement standard will be different for another measurement team.

This can easily lead to completely different evaluations of the same summary results due to the different evaluation teams when judging the accuracy.

The judging teams are very different, and it is easy to cause some teams that are clearly capable of doing well in algorithms to die because of the judging team's overreach.

Eve Carly's team's text summarization algorithm was once the world's leading algorithm.

It has a lot to do with their in-depth cooperation in linguistics with Oxford, Harvard, and Yale universities.

But this is not a long-term solution after all, and the manual evaluation method is doomed to not go far because of its inherent limitations.

Therefore, the text summarization algorithm research team is actively researching automatic evaluation methods.

Since the end of the 90s, some conferences or organizations have begun to work on formulating abstract evaluation standards, and they will also participate in the evaluation of some automatic text summaries.

Well-known conferences or organizations include SUMMAC, DUC, TAC (Text Analysis Conference), etc.

Although related teams are actively researching automatic evaluation methods, the two methods for evaluating the quality of automatic text summarization (manual evaluation method and automatic evaluation method) are still the most commonly used evaluation method.

The principle of many automatic evaluation methods is to compare the news summary generated by the summary algorithm with the reference summary to evaluate by the maximum degree of fitting.

Although the evaluation process is automatic, the reference abstract is written manually.

That is to say, even the so-called automatic evaluation method cannot get rid of the intervention of subjective factors.

In that case, why bother to use an automatic evaluation method?
It is precisely because of this that many teams still choose manual evaluation when evaluating the quality of abstracts.

However, it is difficult to objectively quantify the results of subjective things such as manual evaluation.

Because of this situation, although the accuracy of many team summary algorithms was not bad before.

But when it comes to the promotion of the accuracy of news summaries, everyone selectively forgets.

In this case, why did the developer of Nanfeng APP say in the software introduction that the accuracy of this software is 270% higher than that of similar software.

What standard is this so-called 270% based on?For a moment Eve Carly fell into deep thought.

No matter how this 270% is obtained, it should not be created out of nothing.

Eve doesn't know what the rules are for software promotion in other countries, but in the United States, if there is no logical and self-consistent measurement model as a theoretical support, if you rashly carry out such out-of-the-box quantitative promotion, you will be fined easily. left.

That is, the so-called "270%" high probability of Nanfeng APP is based on a sufficiently powerful and logically self-consistent accuracy measurement model.

But it’s hard to say, every year there are many developers who disregard the rules of publicity in order to gain attention.

Out of a rigorous scientific research attitude, Eve Carly searched with the keyword [Measurement Model of Text Abstract Accuracy].

In the search results, Eve Carly saw at a glance that there was a new accuracy measurement model mixed among the models.

No way, it’s hard not to notice that there are more than a dozen models that measure the accuracy of text summarization in the past, and it’s not an exaggeration for Eve Carly to say that they are so rare.

Now this is called "LH Text Summary Accuracy Measurement Model" Eve Carly has never seen it before.

Take a look at the accuracy measurement method used by this model.

Eve unexpectedly discovered that through this new accuracy measurement model, evaluators do not need to introduce any subjective factors into the process of evaluating the accuracy of abstracts.

Because there is no intervention of subjective factors, this accuracy evaluation method can completely conduct quantitative analysis on the abstract accuracy of all existing text summarization algorithms.

This measurement model also demonstrates several usage examples.

After the algorithm in the Nick Yahoo News Digest software was measured by the model, the accuracy score was only 1 point.

The Nanfeng APP scored 3.7 points.

Seeing this result, Eve understands how Nanfeng APP's so-called summary accuracy is 270% ahead.

It seems that this LH text summarization accuracy measurement model must have been done by the developer of Nanfeng APP.

Even if it wasn't done by Nanfeng APP developers, there should be some kind of connection between the two.

Otherwise, how could the measurement results of this model be highly homogeneous with the data promoted by Nanfeng APP's software.

I have to say that this brand-new method of measuring accuracy called LH gave Eve Carly a feeling of enlightenment.

By using this measurement model, their future research will also be smoother.

However, what surprised Eve Carly was that the "LH Text Summary Accuracy Measurement Model" did not appear in the form of a paper alone.
Instead, it appears in a patent titled "Algorithms for Generative Text Summarization."

Models of measurement appearing in patents?It undoubtedly means that even if this model is very efficient, it still needs to be authorized by the patent owner in theory when it is actually used.

Isn't this too doggy?There is no reason to put this model in the patent.

And is it necessary to apply for a patent just for an algorithm?
Although Eve Carly's previous algorithm was very powerful, they did not apply for a patent.

But Eve had nothing to say about that.

The reason why they don't apply for algorithm patents is not because they are altruistic.

It's because their previous algorithms were only improved on the basis of their predecessors, and they were not completely original.

In addition, applying for a patent will more or less involve a certain degree of technical disclosure.

Although the patent applicant may not disclose all the details, even if the details are not disclosed, the technical route needs to be explained.

Knowing the technical route, the world's top R&D teams are not vegetarians.

Although it is impossible to develop an identical algorithm according to the technical route described in the patent, it is a clear infringement.

However, inspired by the thinking inspired by the technical route disclosed in the patent, it is easy to overtake other similar technologies in corners.

In fact, because of concerns about the leakage of technical routes, there have been few specialized algorithm patents in the United States in recent years.

Um, or is it that this patent owner is so confident that he is not afraid of being chased by others?
Eve saw that the owner of the patent "Generative Text Summarization Algorithm" is Lin Hui
From the spelling point of view, it seems to be a Chinese name, and Eve is ignorant of this name.

However, by searching Lin Hui on Google, Eve found a lot of relevant information easily.

However, none of this information is good news for Eve.

Eve saw that although Lin Hui proposed the "LH Text Summary Accuracy Measurement Model" in the patent.

But he doesn't appear to have plans to make the model private.

Instead, it took the initiative to submit this model to the National Standards Committee of the United States and the International Organization for Standardization for review.

That is to say, Lin Hui not only does not mind making this evaluation method public, but is committed to using this measurement system as the standard for measuring the accuracy of abstracts in the news abstract industry.

It is also understandable, who doesn't want a frame that he made casually to become a common standard all over the world?

In addition to the LH model, there is almost no model for measuring accuracy that does not require the introduction of subjective factors in the news summary industry.

In this case, this "LH text summarization accuracy measurement model" will most likely become the only objective standard for the accuracy measurement of text summarization.

What is this concept?As the saying goes, the first-rate team makes standards, and the second-rate team makes technology.

When Eve and his team were still doing algorithm research at the technical level.

The really ambitious developer Lin Hui not only set out to get a more efficient text summarization algorithm.

At the same time, it also seeks to unify the industry standards.

So did they lose from the start?

Although it has always been calm, Eve Carly can't help but feel a little sad at this time.

She silently engraved the name of Lin Hui, an extremely confident and far-sighted Chinese, in her heart.

(End of this chapter)