tl;dr.: The current AI development is a nightmare for lecturers giving out homework. Or is it?

Days of the Sentients

Okay, here's the thing: AI is not something new. It's way past Gartner's Hype Cycle now, all the way through the Plateau of Productivity. This means it already changes our daily lives, whether we're aware of it or not. AI is now everywhere, and we're taking it for granted. If you unlocked your phone with your face to read this sentence, then you already used one of them.

Of course, one of the most productive driving forces behind the research and adoption of AI into the state that we're seeing now is academia. Conferences such as the recent NeurIPS spew out state-of-the-art reports on AI implementations and evaluations, novel deep learning models, and whatnot. We're literally talking about such exponential growth that it already breaks Moore's Law a couple of years ago!

That being said, on the other side of academia, where lecturers lecture and students, well, study, it actually grows some concerns. Given all that AI is now capable of, it's not surprising that we'd want it to take over tasks that we'd rather not do ourselves. For students, what is more mundane than actually doing your homework?

And to some degree, it has already happened. Students now have an arsenal of tools to help them excel (or worse, cheat) and graduate with flying colours. Aside from some of the most obvious tasks, we now have AI that could solve our math problems, paraphrase our writings, write our hardest code in any programming language, suggest relevant research papers to help with our writer's block, or even better, read the papers for us and churn out the summaries!

So what? We have these tools that will help us make a better assignment, write some new ideas for our research paper, or draw some pretty pictures out of the blue. At some point, it still needs our presence to put two and two together. Rest assured, lecturers! Our students would still need to use their brains to work on our assignments. We can't just ask AI to answer questions from the homework assignments and give fulfilling explanations in plain human language.

Or can we?

Enter ChatGPT

Language-based AI apps have been around for quite some time. What that means is that we could actually chat with the AI, and they would understand what we're actually talking about and respond accordingly. This is possible thanks to the many research efforts in Natural Language Processing (NLP), with one of the most advanced models developed by OpenAI called GPT-3. It was trained with about 45 TB of text data and contains approximately 175 Billion trainable parameters. The sheer amount of training data means that it is capable of engaging in a conversation with a human being on a multitude of topics.

Meanwhile, just a little over four days ago, this particular tweet turned the Twitterverse into turmoil:

What's so special about this ChatGPT?

On the surface, ChatGPT is just another chatbot which is capable of chatting with a human 'in a conversational way'. It is said to be build on GPT-3.5, which offers a glimpse of what GPT-4 would be capable of. However, everyone soon finds out that there's more than meets the eye.

Let's see some examples of what they are capable of doing:

1. Write a movie script:

   https://twitter.com/DenysTsvaig/status/1598955065032986624

2. Write some code:

   https://twitter.com/gabe_ragland/status/1598068207994429441

3. Fixing bugs:

   https://twitter.com/amasad/status/1598042665375105024

4. This (whatever this is):

   https://twitter.com/goodside/status/1598129631609380864

5. Write a classical music piece:

   https://twitter.com/bentossell/status/1598417187177463809

6. And of course, write an academic essay out of thin air:

   https://twitter.com/corry_wang/status/1598176074604507136

And a whole lot more of creative things that anyone could think of. After all, It's free!

Well, it is capable of doing so much things. It don't take long before students are taking advantages of using it for doing their homeworks. After all, it's really good at doing just that:

Got a homework and a minute? Maybe it's worth typing some sentences to ask AI to do that for you, and et voila, it's done!

Break Stuffs and Play Around

Apparently, ChatGPT is very good at answering coding-related problems. So obviously I need to do some 'testing' (or rather, playing). So far, the results are quite pleasing.

For example, this is a rather common question that I often give as assignment to students:

Well, not quite what I expected, but it did give a perfectly working Python code for the problem.

Satisfied? Well, not really. I would expect such answer from most of the average students, but it should do better. In fact, ChatGPT already mentioned it in the answer: for better performance, use dynamic programming. Since ChatGPT has a 'long-term memory' (i.e., it remembers our previous chat in the session), I asked them to revise the answer:

And here's the code that it printed out:

def fibonacci(n, memo):
  # Base case
  if n == 0 or n == 1:
    return n

  # Return previously computed value if available
  if memo[n] != -1:
    return memo[n]

  # Calculate the nth value in the Fibonacci sequence
  memo[n] = fibonacci(n-1, memo) + fibonacci(n-2, memo)

  return memo[n]

# Initialize the memo array with -1 for all values
memo = [-1] * 101

# Print the 10th number in the Fibonacci sequence
print(fibonacci(10, memo))

Surprise surprise! It provided the code to memoize the sequence, and gave a great example of how it should be done. And if this was a student assignment, they would just copy this answer for explanation and got some good score:

Wait, that's just a generic coding-related questions. What about some more domain-specific questions?

Alright, this is from another quiz:

This is the code from the answer:

-- First, create a table to store the locations of schools
CREATE TABLE schools (
  id SERIAL PRIMARY KEY,
  name VARCHAR(255),
  location GEOGRAPHY(POINT, 4326)
);

-- Then, create a table to store the locations of cafes
CREATE TABLE cafes (
  id SERIAL PRIMARY KEY,
  name VARCHAR(255),
  location GEOGRAPHY(POINT, 4326)
);

-- Next, insert the data for the schools and cafes into the respective tables
INSERT INTO schools (name, location) VALUES
  ('School A', ST_MakePoint(-0.1275, 51.507222)),
  ('School B', ST_MakePoint(-0.141667, 51.493333)),
  ...;

INSERT INTO cafes (name, location) VALUES
  ('Cafe A', ST_MakePoint(-0.13, 51.507778)),
  ('Cafe B', ST_MakePoint(-0.141111, 51.498889)),
  ...;

-- Finally, use the ST_DWithin function to find the 5 nearest cafes
-- from each school and return their names and distances
SELECT
  s.name as school_name,
  c.name as cafe_name,
  ST_Distance(s.location, c.location) as distance
FROM schools s
JOIN cafes c ON ST_DWithin(s.location, c.location, 5000)
ORDER BY s.name, distance
LIMIT 5;

And the explanation:

This query first creates two tables called schools and cafes to store the locations of schools and cafes, respectively. The schools and cafes tables have a name column that stores the name of the school or cafe and a location column that stores the geographic coordinates of the school or cafe.

Next, the query inserts the data for the schools and cafes into the respective tables. The location column uses the GEOGRAPHY data type and the ST_MakePoint function to store the coordinates of the schools and cafes.

Finally, the query uses the ST_DWithin function to find the 5 nearest cafes from each school. The ST_DWithin function takes two points and a distance as arguments and returns true if the two points are within the given distance of each other. In this query, the ST_DWithin function is used to find cafes that are within 5 kilometers of each school. The query then returns the names of the schools and cafes, as well as the distance between them.

Note that this is just one possible way to solve this problem using PostgreSQL and PostGIS. There may be other ways to accomplish the same thing using different SQL commands or PostGIS functions.

With some very little tweak here and there, it would strike a perfect score for the question.

Still not convinced? Here's another one:

(did I mentioned that apparently it could understand different languages too?)

Here's the code that got printed:

% Define the constants
g = 9.81; % gravitational acceleration (m/s^2)
omega = 7.2921158553e-5; % angular velocity of Earth (rad/s)

% Read the data file
data = readtable('tide_data.csv');

% Extract the time and height data
time = data.time;
height = data.height;

% Fit a sine wave to the data using least squares
A = [sin(omega * time), cos(omega * time)];
b = height;
x = A \ b;

% Calculate the amplitude and phase of the sine wave
amplitude = sqrt(x(1)^2 + x(2)^2);
phase = atan2(x(1), x(2));

% Calculate the tidal constants
k1 = 3 * sqrt(2 * g / (omega^2 * amplitude^2)) * sin(phase);
k2 = 3 * sqrt(g / (omega^2 * amplitude^2)) * (3 * cos(phase) - 1);

% Print the tidal constants
fprintf('K1 = %.5f\nK2 = %.5f\n', k1, k2);

And the explanation:

It's kinda flat, but it's understandable, given that I only gave them very limited data to begin with.

Another one:

Not convincing. But again, a student might submit this as assignment, and an unassuming lecturer might give some reasonable score for the work.

Out of curiosity, I tried to ask them to write a poem:

...and it failed miserably 😒. I guess they still have a lot to learn about Chairil Anwar's poems.

Are.We.Doomed.Yet?

Are these answers by ChatGPT always correct?

Well, in this case, not always.

It is quite good at coding related problems, and math, and biology, and chemistry and a lot more things. In fact, one of my colleagues use ChatGPT to build a deep learning model, which helps improve the code he's been writing (!). However, it does have some limitations. As ChatGPT themself put it:

As a large language model, I have been trained on a vast amount of text data and can generate human-like text on a wide range of subjects. However, I am not an expert on any specific topic and do not have specialized knowledge beyond what I have learned from the text data I was trained on.

For some questions related to specific person, or events in history, ChatGPT refuses to give answers at all. It might be helpful for solving most questions, but not all. However, rather than expecting a perfect answer everytime, we might get better results by tweaking the questions incrementally, just like what I did with the Fibonacci question. This way, a student can take advantage of getting some help with their homework. Take it as a brainstorming tool to better understand the subject, and start from there to finish your homework on your own.

So don't worry. The AI won't take over the world at this very moment. Fingers crossed, this Terminator scene won't be happening any time soon.

Would it be cheating for a student to use these AI tools? I personally don't think so.

Should we stop students from using these tools?

Maybe we shouldn't. Or rather, we couldn't.

One way or another, these tools will find its way in the hands of our students. In fact, it would affect not only students doing their study: it might as well change a lot of what we're doing as a lecturer. Take, for example:

Basically, the lecturers could do the same: we could use the AI to enrich our teaching materials, and who knows? It might be useful to ask one or two exam questions 😁.

And for the students out there reading this:

Don't confuse AI's confidence with being correct

The AI might seems confident with their answer. But unless you already know the right answer, you won't be able to tell the difference. Never trust the result blindly. The danger of trusting too much is that you never know till you learn your lesson.

So use this tools with caution. After all, that's what you're learning for: to think critically, to question scientifically and to write academically.