Crossover: 2014

Chapter 258 Huge Invisible Wealth

What Eve Carly calls "content representation" refers to the process of dividing raw text into text units in the process of automatic text summarization.

This process includes preprocessing work such as word segmentation, words, and sentences;

Its main purpose is to process raw text into a form that can be easily analyzed by algorithms through preprocessing.

Traditional automatic text summarization such as traditional extractive summarization does not pay much attention to the link of content representation.

Generative text summarization is slightly different from traditional extractive summarization in this part.

Generative text summarization still pays more attention to the content representation.

In particular, the generative text summarization using word embedding technology and pre-training mechanism pays special attention to the "content representation" link.

No way, I have to pay attention to the fact that the importance of each step of traditional text summarization is almost the same.

However, the work of generative text summarization using word embedding technology and pre-training mechanism is often "top-heavy".

That is, the beginning link has the highest weight in the whole link.

In other words, when actually building a generative text summarization model, although there are many steps to be designed.

But usually the more advanced jobs are often more important.

Let's take "content representation" as an example, when building a generative text summary model.

In many cases, the completion level of the content representation will directly affect the subsequent steps.

And the patent "A New Method for Text Judgment, Screening and Comparison" made by Eve Carly does have certain value in terms of content representation.

With the help of the value provided by this patent, Lin Hui can reduce some logical loopholes in the process of subsequent upgrading and updating of text summaries.

But if it is only because of this value, it is not enough for Lin Hui to spend a lot of time to acquire a cross-border patent.

The most fundamental reason why Lin Hui took great pains to obtain the patent "A New Method for Text Judgment, Screening and Comparison" created by Eve Carly is because Lin Hui is more concerned about Eve Carly's patent in this patent. The model to apply.

In the patent "A New Method for Text Judgment, Screening and Comparison", regarding text screening, Eve Kali creatively tinkered with a model for text judgment and screening.

If only in terms of natural language processing machine learning, this is just a mediocre model for text discrimination.

But when the thinking jumps out of the small field of natural language processing, this model can't be taken lightly.

When browsing through some academic materials of this time and space, Lin Hui paid keen attention to the value contained in this patent.

Although the technical routes provided by patents are often general.

Some latecomers often can only explore like a blind man and an elephant when they follow these technical routes to understand the technology.

With the information of his past life, Lin Hui is equivalent to standing on the shoulders of giants,
Although occasionally there is a feeling of being overwhelmed by high places, but when it comes to technology, Lin Hui often has a stronger system concept.

In many cases, Lin Hui only needs to see some public technical routes to understand the value behind them.

And this kind of judgment is basically inseparable.

When I first came into contact with Eve Carly for this patent.

Lin Hui found that according to some information disclosed in the patent, especially the technical route mentioned in the patent disclosure.

Lin Hui quickly grasped the value of this patent.

Lin Hui decided to use this model to form a very efficient discriminant model on this basis with almost a slight deformation.

The fact is that after further understanding of the patent information in the subsequent acquisitions, it confirmed Lin Hui's previous guess about it.

Just discriminative models may not make much sense even if they are efficient.

But if you make a small change, things will be different.

When an efficient discriminative model meets an efficient generative model.

The two are organically combined, and on this basis, a certain special structure is continued.

It is entirely possible to use this to create a new and highly efficient deep learning model.

This deep learning model had a famous name in its previous life:

— Generative Adversarial Network (GAN)
Generative adversarial networks consist of a generative network and a discriminative network.

The generator network randomly samples from the latent space as input, and its output needs to mimic the real samples in the training set as much as possible.

The input of the discriminative network is the real sample or the output of the generation network, and its purpose is to distinguish the output of the generation network from the real sample as much as possible.

The generative network should deceive the discriminative network as much as possible.

The two networks play against each other, constantly adjusting parameters.

The ultimate goal is to make the discriminative network unable to judge whether the output of the generating network is real or not.

In an academic forum, Yann Le Cun, the former Turing Award winner and the father of convolutional neural networks, even called the generation of confrontation network model the coolest idea in machine learning in 20 years.

Being highly affirmed by a Turing Award-level leader, the value of generating an adversarial network model can be imagined.

Past Life Generative Adversarial Networks as a Method for Unsupervised Learning.

It was proposed by Ian Goodfellow et al. in 2014.

However, this time and space is lagging behind as a whole due to the research on machine learning.

It seems a bit difficult for this well-known deep learning model from the previous life to come as promised in this time and space.

Since the emergence of past-life generative adversarial networks, many variants have appeared for different application fields.

These variants all offer certain improvements over the most primitive generative adversarial networks.

Some of these improvements simply improve the structure.

Some of them have improved some functions or parameters involved in the generative confrontation model because of theoretical development.

Or simply make some innovative adjustments in terms of application.

Frequent changes to a technology does not mean that the technology has failed.

On the contrary, it just means that the technology is successful.

Because this reflects to some extent from the side that the technology has a lot of room for growth.

The fact is exactly the same, the previous life generative confrontation network is quite successful and widely used.

Generative adversarial networks can be seen in many fields of machine learning.

The reason for this is probably because the original generative adversarial network was constructed with fewer a priori assumptions.

It is precisely because there are almost no assumptions about the data that the generative adversarial network has almost unlimited modeling capabilities.

With the help of generative adversarial networks, various distributions can be fitted.

In addition, due to the low complexity of the generative adversarial network model.

In many cases, there is no need to pre-design more complex function models when applying generative confrontation networks.

In many application scenarios of generative confrontation networks, engineers even only need to apply the backpropagation algorithm to simply train the corresponding network.

This allows the generator and discriminator in the generative adversarial network to work normally.

The reason why the generative confrontation network is so easy to use.

It also has a lot to do with the original design of the generative network for unsupervised learning.

But everything has two sides, precisely because the original generative confrontation network is too free.

The training process is prone to training divergence.

More than that, the generative confrontation network also has problems such as gradient disappearance.

Due to these problems, it is difficult for generative adversarial networks to learn some generative discrete distributions.

For example, the original generative confrontation network is not very good at processing pure text.

In addition to the use of generative confrontation networks for text segmentation in some scenarios.

Most of the time, generative adversarial networks are rarely applied to text (especially text in pure text form).

However, the ruler is long and the inch is short, although it is not very good at processing pure text information.

But in many other fields, generative adversarial networks can show their talents.

In terms of face recognition, super-resolution reconstruction, etc., generating adversarial networks is even more useful.

Even generative adversarial networks can show their talents in semantic image restoration.

In addition, there are many application directions of generative confrontation network.

In a nutshell, the application prospects of generative adversarial networks are quite broad.

Speaking of it, the research on this space-time machine learning is relatively lagging behind.

If Lin Hui wants to transfer the model of generative confrontation network, he doesn't need to take too many risks.

However, before the generative text summarization is completely done.

Lin Hui is not in a hurry to carry out the research results related to the generative confrontation network.

As for why Lin Hui didn't move out the generative confrontation network?
Lin Hui didn't want to give the rest of the academic staff a sense of isolation.

Just like Lin Hui did not want to give game players a sense of fragmentation when developing (transporting) games in the past.

Although Lin Hui now has a certain logical basis for the launch of the generative confrontation network.

(Lin Hui previously worked on the generative model involved in the generative text summarization, and the patent acquired from Eve Karina involved the class discriminative model, and the composition of the generative adversarial network contains the generative network and discriminative network...)
But if Lin Hui rashly comes up with a generative confrontation network, it's still not very good.

After all, in terms of application level, the generative confrontation network has little to do with the academic field of natural language processing that Lin Hui has been working on.

In this case, why does Lin Hui inexplicably launch a model that has little to do with natural language processing?
Although there are many examples of unintentional academic achievements, the purpose of many academic achievements is often deviated when they first come out.

However, the principle that Lin Hui believes in is destined to make Lin Hui unlikely to break the previous practice.

Whether it's game development or academic progress, Lin Hui doesn't want to give others a sense of isolation.

Moreover, it is better to point the technology tree in order.

Although it is okay to click the technology tree out of order as a hanging ratio.

But in a pluralistic society, not playing by the rules often means risk.

If you mess up the technology tree, your own technology logic chain has not been formed.

Potential opponents have formed a corresponding development context.

Then the scientific and technological achievements are likely to be stolen by the opponent.

This is what Lin Hui doesn't want to see.

Now, in Lin Hui's view, what he has to do academically is still to cultivate natural language processing.

Deep cultivation of generative text summarization.

Through continuous deep cultivation, find a breakthrough point in the field of natural language processing

In other words, it is best to light up the branches of the technology tree that are adjacent to the forest ash that has already lit up the technology achievements.

(Lin Hui is not in a hurry, even if he doesn't find a suitable breaking point for a while, it doesn't really matter.

At least for a month, Lin Hui doesn't need to worry too much.

After all, the "breakthrough progress (successful handling)" Lin Hui has achieved in terms of generative text summarization can at least "mix" a master's degree.

And it will take some time for Lin Hui to digest it.

In fact, Lin Hui's original estimate was more optimistic.

Lin Hui originally thought that if he understood the thesis in the direction of generative text summarization, he would almost be able to get a Ph.D.

However, through the recent communication with Eve Carly, Lin Hui felt that he was too optimistic.

Just like a Nobel Prize-level achievement does not necessarily win a Nobel Prize.

Even if Lin Hui's work on generative text summarization can be regarded as a doctoral level or even higher level achievement for this time and space.

But it is also very difficult to obtain a doctoral dissertation in one step.

After all, the main presentation form of Lin Hui's academic content was around an algorithm patent such as generative text summarization.

In this time and space, the West tends to regard academic achievements in the form of patents as something more practical, that is, engineering achievements.

It is very troublesome to rely solely on engineering achievements to obtain doctoral achievements in one step.

Although the academic benefits related to generative text summarization are slightly lower than Lin Hui's expectations, it is not a big problem.

Lin Hui felt that it was not entirely a good thing to go too far academically. )
Since the generative confrontation network will not be transferred in a short time.

Isn't the thinking about the generative confrontation network just now equivalent to wasting brain cells in vain?

of course not.

In many cases, the thinking probably gets new inspiration in some casual thinking.

Thinking about the generative confrontation network, Lin Hui suddenly realized that he still had a huge hidden wealth.

That is the artificially labeled data from the previous life.

Although I didn't look too carefully at the information I brought with me in my previous life.

However, it is impossible for Lin Hui to have manually labeled data.

Especially those enterprise-level hard drives in the previous life are absolutely impossible without manually labeled data.

Even if there is no manual labeling of images, manual labeling of some texts is absolutely impossible.

After all, this kind of thing is quite practical, and the text annotation does not take up much space.

You must know that when it comes to neural network learning training or deep learning training, a large amount of manually labeled data is required when the model is built.

In particular, supervised learning and semi-supervised learning require a large amount of manually labeled data.

Usually a model requires a lot of manually labeled data when it is constructed.

When adjusting, a lot of manual labeling data is also required.

(End of this chapter)