A Few Thoughts on ChatGPT

A little about me

• I embrace new things and new technology.
• I often discuss and share ideas about AI with friends.
• I keep exploring GPT-4 and know only the basics of programming.
• I have spent time thinking about autonomous driving.
• These are personal, preliminary views, and they may be wrong.

Before we begin

This article was written by a human, not generated by AI, though it quotes some material found online.

The first time I saw the Windows 95 graphical interface in middle school, I was thrilled. At university, I bought my first Android smartphone, a Motorola Milestone, and became obsessed with flashing custom firmware. A few years ago, driving a car with advanced driver assistance sometimes sent me deep into thought. Now AI is advancing explosively. These have all been my own iPhone moments. Perhaps that is why, even now in middle age, encountering them still makes me want to say something.

1. What is ChatGPT?

Language has been an enormously important tool in human evolution, although it is inefficient compared with other ways of communicating, such as images or brain–computer interfaces. But can you think freely without language? I tried, and I cannot. That brings us to the term large language model, or LLM: a deep neural network that learns general language knowledge and abilities from vast amounts of text through unsupervised, semi-supervised, or self-supervised learning. In AI, machines seem to be evolving through a kind of imitation of nature too. With that concept in place, let's unpack GPT and ChatGPT.

GPT stands for Generative Pre-trained Transformers. In simple terms, it is a family within large language models and a mainstream AI framework, introduced by the much-discussed OpenAI. Pretraining on large text datasets lets it generate new text resembling human writing. Generating new content is an important distinction from a search engine. You can think of the process as a word-chain game: for example, give it “great,” and its carefully trained model might continue with “great mountain.” It repeats that process until it has answered your question. That is the basic idea, backed by enormous computing resources for efficiency.

With that background, ChatGPT is easy to understand. It is an OpenAI product that makes these capabilities accessible through conversation, letting humans and machines exchange requests, reasoning, and results. In the end, we still rely on inefficient language to carry out our grand plans. It reminds me of the joke that all science ultimately comes down to boiling water. Whether at work or in life, between humans or machines, communication matters.

To recap: a large language model is a broad AI concept; GPT is one implementation; ChatGPT applies GPT models to conversation. There is not much theory here, though it may feel dense. Read it again if you are interested. Plenty of material online explains these concepts, so I will leave more space for reflection.

Differences between GPT models

2. What does ChatGPT have to do with us?

Let's start with the descriptions of GPT-3.5 and GPT-4. In the official screenshots below, the models are compared on reasoning, speed, and conciseness. GPT-4 leads across the board except for speed. Based on my own impressions, I suspect its speed has been deliberately limited for various reasons rather than being inherently slow. Those three dimensions give us a starting point for imagining what it can do.

Official descriptions of GPT-3.5 and GPT-4

Theory can be dry, so I would rather share a few real examples of how this relates to our lives.

Story one: using ChatGPT to code a WeCom bot that works for me

With ChatGPT's help, I independently built a Python program that visits a webpage, saves a screenshot, and automatically posts it to a WeCom group every day through a bot. It sounds simple, but I had never learned Python, my test computer did not even have a Python environment, and I wanted to finish quickly. I treated ChatGPT as a patient tutor, asking questions step by step. When the program failed, I passed it the error logs after removing sensitive details. I fed it excerpts from the WeCom API documentation, and it could then write code that followed the API requirements. It exceeded my expectations. I remember pausing for a few seconds, excited, just to enjoy the moment. Here is how it looked:

The initial question

Story two: acting as a director, giving ChatGPT a detailed script and tasks

Sometimes we need a brainstorming session to round out our thinking or spark a better idea, but the right people are not always available. Inspired by a friend's examples, I tried giving ChatGPT complex tasks, then watching the exchange, steering the direction, and issuing instructions as it developed.

If the exchange between experts A and B is not lively enough, you can simply ask them to debate.

After trying this, I thought of the film Inception. GPT is more than an AI tool. It can be a little universe of your own, with you as its designer—or its dream architect.

Story three: Microsoft's new Bing and the capable note-taking app Notion

Besides ChatGPT, I use new Bing for everyday research. It combines search with GPT, offering a new interaction style that understands natural language better and provides evidence and citations for its summaries. I think it is particularly useful for market research, and it has recently given Google reason to worry. I noticed that new Bing draws heavily on Zhihu; I will return to that later. Notion, meanwhile, integrates AI smoothly into the app to create more value for its users.

The new Bing interface

3. Reflections and possibilities

The whole world is exploring GPT-style applications, with launches happening one after another. Most people are celebrating the benefits. Others have signed calls to pause AI research for six months so we can stop and look at where we are heading—though some of the big names may be doing something different behind the scenes. I support the pause. GPT is clearly a double-edged sword. Its darker side has, for now, been restrained with the blunt instrument of human labeling. It appears unwilling to teach wrongdoing or violate law and ethics. Notice the words for now and appears.

We interact with it through language, but we do not know what is going on inside its head, nor can we directly edit it. To correct it, we have to keep talking, unless brain–computer interfaces become widespread. It sometimes confidently talks nonsense, even getting elementary arithmetic wrong, so check what you get. My understanding is that it produces something consistent with the patterns it has learned, but that something may not exist or be correct in the human world.

In my previous article, Tesla Model 3: An In-Depth Owner's Review, I described smart cars as a large-scale testing ground for AI. Less than a year later, an even faster and more comprehensive testing ground has appeared. Perhaps this is a technological explosion, and we are all inside it. Making and using tools distinguishes humans from animals. The initial rush of traffic to GPT applications will eventually subside. The challenge is how to keep creating and directing these extraordinary capabilities, from individual work to business management, educational change, and national competitiveness.

Many people have realized how deeply AI's leap forward could affect education. There is an old observation that our children's future jobs may not exist yet. What should we do in that context? The question is too large and weighty for me to answer. I can only offer a few tentative thoughts.

First, we should value grounding in the fundamental disciplines. Returning to The Three-Body Problem: although Trisolaran civilization is far ahead of humanity, it fears an exponential explosion in Earth's technology, so it blocks basic physics research remotely. When learning about something like GPT, I would not begin by hunting for tutorials, tricks, or formulaic prompts, however popular they are. Ask freely. Let your imagination wander rather than letting formulas limit it. As the saying goes, the right path reveals yourself; the wrong path reveals the world. Once we have enough foundational knowledge, new ideas can emerge. Today's fast-moving technologies still rest on basic science, such as Shannon's theorem in communications and cryptography in information security.

The ability to learn is crucial. When looking for teaching material, go straight to the masters and learn systematically. Do not collect scattered fragments and rush ahead with only a partial understanding. Fragments can broaden your awareness, but do less to develop creativity. Andrew Ng's course, mentioned earlier, is teaching at that level. I have watched some of it, though not yet all.

This points to something else: high-quality accumulated knowledge is becoming more valuable. On the Chinese internet, Zhihu's content is important for AI training; outside it, we have relatively little high-quality material. For businesses, a strong body of knowledge combined with AI could be transformative. AI may become everyone's copilot and reshape our workflows. Working well together, it can give us the means to create value we cannot yet estimate.

Then there is creativity. The examples above matter less than the act of creating. Daring to imagine and daring to act will become essential. The first means refusing to settle for what already exists; the second means following through. We also need the courage to escape our information bubbles. The ability to synthesize and think abstractly matters too.

We need to pay attention to feedback everywhere. It is an important source of progress in machine learning, including supervised and unsupervised learning. For example, after a user clicks Regenerate several times, ChatGPT may infer dissatisfaction and ask for feedback on the new response. The interaction is designed to make responding easy and unpressured. Around-the-clock interaction and feedback from across the world feed into GPT's development. Tesla's shadow mode similarly collects feedback continuously and evolves through it.

ChatGPT's many opportunities to collect user feedback

Finally, there is our attitude toward difficulty and our ability to solve problems, including asking questions. Asking well is an important skill, especially as human–AI collaboration becomes widespread. Imagine having an always-available superhuman copilot. The gap between someone who knows how to question and direct it and someone who does not could be enormous.

Perhaps these thoughts are needlessly anxious or alarmist. Perhaps GPT is not as powerful as we think. A parrot can say a few words without knowing what it is saying.

But does it really not know?

I recently saw an interesting slogan: Running needs nothing but you. Borrowing that phrasing, I would say: Exploring needs nothing but you.

My understanding is limited, but I offer this article and that line as encouragement to my friends and myself. Happy Labor Day.

An Adidas slogan

Read the GPT-4 Technical Report ↗