Nearly thirty years old, established in science and technology

Chapter 57 Sun Bo’s Ability

Microsoft and OpenAI did not keep the large-scale purchase of graphics cards secret at all, and even made it extremely high-profile. As soon as the decision was made here, it was immediately known to everyone around the world.

"This..." Ji Hong was dumbfounded when he saw the relevant news: Is the prediction model so accurate?
It's outrageous to think about, but after careful analysis, it does make sense - because Microsoft has no second way to go.

Although the Whisper system does not show super attack power, and Juaner Intelligent Technology did not even hold a press conference for it, and it seems that it does not take it seriously, its combat power is there, and no one can ignore it.

Even if it is not officially sold in the United States and Europe at all, it has begun to slowly invade Microsoft's Windows market share.

If the situation cannot be reversed in the next version, the disadvantage will turn into defeat in an instant!
It is true that ecology is a moat, but there is a precedent for people being drowned in the city by moats - Nokia's Symbian system had a high market share back then, right? Now look again, where can you see its shadow!

The moat is for products of the same level. If it falls too far behind, you will only fall into the river.

Ji Hong also thought of this, and then looked at the predictive AI model: If there are too many variables and the time is long, it will be really difficult for it.

But if there are few variables, like this time, predicting the GPU industry, just for this industry alone, seems to be very reliable to predict the short-term trends of certain companies.

"This thing can be a very powerful product if packaged!" Ji Hong thought: "Let's give it to Chen Jiye for business analysis first."

"Microsoft here..." He was so determined that everyone could see what Microsoft was going to do, and Ji Hong was no exception. At this time, he was thinking about this question:
"Now their large models, such as Transformer, can infinitely improve the model capabilities by relying on unlimited accumulation of computing power?"

Ji Hong has really not done any research on this - before, in Chuangjie, he mainly used other platforms for interface docking development, and later started his own business, using a class thinking model, which was a dimensionality reduction attack on the T model, and there was no such problem.

He has no research on this, but there is one person who has been paying attention to this aspect - this person is Sun Bo.

"It is feasible in the short term to obtain high-performance large models by relying on unlimited accumulation of computing power."

Sun Bo also saw Microsoft's actions and was discussing related topics with a group of people on CSDNBlog:
“In the long term, there is no upper limit. If you really have the courage to accumulate computing power without any limit regardless of cost, in theory, you can really achieve unlimited performance improvements.

"But that's just in theory. There's a huge problem here. According to my research, its growth is not linear.

“I have roughly calculated it. I won’t talk about the actual growth curve. It is very complicated, but it can be roughly simplified to this function: y=c·log(e)·(X+1).

"Y here refers to performance, and X can be understood as the basic unit of computing power accumulation. The approximate trend is as follows."

Even people who don't know much about mathematics should recognize this formula when they see it, because the curve of this function is almost a flat horizontal line at the end.

In other words, when the accumulation of graphics cards reaches a certain scale, even if more hardware is accumulated, the performance will increase, but the increase will be very limited, and in the end it will not even be visible to the naked eye.

Having said this, Sun Bo also gave an example: "Take the current GPT4 as an example. If you want a large model with 2 times the parameters of GPT4, how much computing power will be required?

"I have calculated that it takes about ten times. Moreover, the formula only calculates the theoretical effect. For larger-scale clusters, the loss of inter-chip interconnection will definitely be greater, and the actual effect will only be worse."

Sun Bo is very enthusiastic. As long as it does not involve the secrets and core of his own work, as long as he is around, he will answer almost all questions.

Ji Hong also happened to read this, and he was not too surprised. The loss of inter-chip interconnection itself is difficult to solve, otherwise ultra-large-scale intelligent computing centers would not be so rare - the one developed by Hezhou University is still The first purely domestic example.

"According to what you say, the effect of Microsoft's all-out effort may not be good!" someone replied.

"It is certain that the effect is not good. At twice the price, you may get 1.5 times the performance, but that's it at most. I don't think it's feasible to catch up with Whisper's ability." In fact, OpenAI must also know that, The reason why GPT4 and even the unreleased version 4.5 are not so large is because of the two nodes with poor energy efficiency.

"As the scale continues, the price-performance ratio will become lower and lower. But aren't we being forced to do nothing by the whispers now?"

Sun Bob was originally studying cutting-edge theory. He knew the theoretical upper limit of existing models very well, and he probably also knew which computing power node was the most cost-effective.

To a certain extent, blindly increasing the scale will definitely outweigh the gains. Unless absolutely necessary, I generally wouldn't do this.

Ji Hong looked at Sun Bo's reply while analyzing this person. The more he analyzed, the more interested he became - this is a theorist with real materials.

"If we can get the ear-curling technology, it will be a huge help."

……

When Ji Hong was talking about Sun Bo, Sun Bo was also talking about him - the reason was the private message Ji Hong sent.

"A branch of optimization theory, the purpose is to solve the problem of communication between large models. Could this person be Ji Hong, the founder of Juaner Intelligent Technology?"

Sun Bo looked at it for a long time and thought of what Ji Hong said in an interview at Hezhou University. The combination of two, three, and more models can produce more accurate and more relevant results. Let me tell you first.

"Is this an idea for realizing AGI general artificial intelligence?"

Sun Bo was a little confused because Ji Hong had omitted many key points for him, but the general framework was still there. He knew what problems this thing was solving.

In fact, he himself is working on this.

"However, communication between large models is based on data and information. Generally speaking, communication only expands the amount of information."

For example, you can use the data of one large model to quickly train another large model - as everyone knows, Qianxun's Wen Xinyiyan used GPT data.

Later, it was revealed that Google's large model had used Wen Xinyiyan's training data, and it was not uncommon for everyone to use it.

"But this still requires a training process. It just lacks the data cleaning and calibration process and related labor costs. The capabilities of the two models cannot be directly added. All that can be used is data!"

"Unless..." A bold idea gradually emerged in Sun Bo's mind: "Unless there is a way to directly communicate or even integrate intelligent abilities. This..."

Although it was just an idea, Sun Bo was also shocked by himself: What if the intelligence capabilities of the two models could be integrated and used together, but is this possible? This is equivalent to communication between people!
"Thinking..."

Just now I thought it was impossible, but as soon as the word "similar thinking" came out, Sun Bo's head immediately buzzed. There was already a precedent: "This person is definitely Ji Hong who didn't run away. He is interested in me." ?”

　Please read it, especially the chapter about tomorrow and the day after tomorrow. It is related to whether this book can break out of the classification and whether it can compete with Sanjiang.

　　Therefore, I urge everyone to read the latest chapters in the past two days. Thank you very much!

　　
　
(End of this chapter)