First GPT

I am starting by exploring some of the tools available and a natural place to start is Chat GPT. The goal is to simplify and break down homework assignments so a student does not get lost in the language. I want to provide a clearer picture of what is being asked.

To start, I thought I’d try a math word problem. A huge part of a word problem is to present a life-like situation to see how math can apply to everyday situations. At the same time, there is also a lot of nuance in language that can get in the way of transmitting information. Here is the word problem I am using as an example.

For this year’s harvest festival, farmer Bob is aiming to to win the grand prize. Fred, last year’s winner, broke the record by harvesting seven bushels of gourds. Every bushel can hold seven gourds. How many gourds does farmer Bob have to harvest to break Fred’s record?

The math involved is appropriate for a 3rd grader. Although the average 3rd grader might not be familiar with gourds or bushels, they can probably figure out what the problem is asking. However, for some students it might be hard to get past all the details in the problem and the writing style is almost purposefully trying to make it harder to understand.

I decided to create a GPT called Communication Guide that serves as a helper. Here is what is currently in the instructions section:

Communication Guide is designed as a supportive speech therapy tool, primarily for children with language processing challenges such as those associated with Autism Spectrum Disorder or Dyslexia. It specializes in breaking down language and illustrating concepts in homework assignments, particularly word problems, making them more accessible. Importantly, Communication Guide avoids solving homework for users. Instead, it focuses on dissecting the language of the problems, explaining terminology, and clarifying concepts, empowering children to understand and solve the problems on their own. The illustrations provided are simple, using basic shapes and flat colors for clear understanding. The language used is clear, concise, and engaging, ensuring accessibility for children with nuances in language understanding. This approach encourages independence and learning, guiding users through understanding word problems’ components, but not providing direct answers.

These instructions were auto-filled after a few prompts that clarified what I needed. Here are the results of running my problem through Communication Guide. The prompt was “Help me with this problem” and the problem as specified above. Here is the result:

Sure, let’s break down this word problem together!
Understanding the Goal: Farmer Bob wants to win the grand prize by breaking last year’s record.
What is the Record? Last year, Fred won by harvesting seven bushels of gourds.
What is a Bushel? A bushel is like a big basket. Each bushel can hold seven gourds.
Fred’s Total Gourds: Since Fred harvested seven bushels and each bushel holds seven gourds, we multiply these two numbers together to find out how many gourds Fred had. That’s 7 bushels x 7 gourds per bushel.
Breaking the Record: To break the record, Farmer Bob needs to harvest more gourds than Fred did.
So, your task is to multiply the number of bushels by the number of gourds in each bushel to find out how many gourds Fred had. Then think about what number is more than Fred’s total to know how many gourds Farmer Bob needs.
I’ll make an illustration to help visualize this. Let’s imagine each bushel as a big basket, and inside each basket, there are seven gourds.
Let’s create the image:
Seven big baskets (bushels) arranged in a row.
Each basket contains seven gourds.
The gourds are simple, round shapes.
Use basic colors to differentiate the baskets from the gourds.
This image will help you see what Fred’s harvest looked like!



Here’s the illustration showing seven baskets, each with seven gourds inside. This visualizes what Fred’s harvest of seven bushels looked like. Each basket (bushel) has seven gourds.
Now, think about how many total gourds are shown here by adding up all the gourds in each basket. And remember, for Farmer Bob to win, he needs to harvest more than this total number of gourds. Can you figure out how many gourds are in the picture and then determine how many Farmer Bob needs to harvest to win? Remember, it’s okay to take your time to count and think about it!

So, kinda-sorta? But also not at all. A big part of the challenge in trying to accomplish my goal by using machine learning is that when the AI gets it wrong, it is really important for it not to make things worse. The initial breakdown seemed to be going in the right direction, but everything after “So, your task…” is the opposite of what we need. Way too wordy. I imagine this can be addressed and corrected by further refining the instructions. This seems to be a good place to start to improve it.

But then there is the illustration. I know image generation still has issues with labels and letters so it will be an interesting challenge to see how close we can get to what we want and whether it’s attainable with the current tools available or if we have to go deeper. But clearly this is a problem since it isn’t at all clarifying the text. Should the student try to approach a solution by actually counting the gourds or reading the symbols and numbers, they would not get anywhere near the solution and would probably feel even more confused and frustrated.

Now, I didn’t actually expect to solve this problem just yet, but I did want to have an initial stab at it to see where my baseline is.

This brings me to my approach. My goal is to develop three different but interlaced tracks to research and develop tools that can support speech and language challenges.

The first is learning about techniques, tools and exercises that are being used by therapists to help children that have speech and language challenges. I have some access to these techniques but it’s going to be important to identify which aspects can be made easier or more effective using technology.

Another is exploring what digital tools are already available in the realm of language processing and how people are using them. I explored a little of that today.

The final one is to learn and understand how these tools are built. I’m grateful that Python seems to be one of the languages that seems to have a lot of support in Machine Learning, so it will be nice not to have to learn a whole new language to move forward with PyTorch. I’m reading through Deep Learning with Python and following-along with Daniel Bourke’s massive Learn PyTorch for deep learning in a day. Literally video. I like having multiple sources of information in different media. Books are great for looking back and reviewing, videos are great for incorporating visual and auditory experience into the learning. Who knows, maybe the tools I develop can help me learn better too. Wouldn’t that be sweet?