nano-GPT train for letter sorting

[Nuclear6]

Great project! I would like to know how to train the letter sorting model mentioned in the nano-GPT visualization project? I think the visualization here is more clear when combined with the nano-GPT project code? So is it convenient to provide training documents for the alphabetical sorting model? Thank you so much!

[Nuclear6]

I am a newbie in AI and want to try it myself, but I find it is still a bit difficult.

[gitzhangzhao]

@Nuclear6

I generated some training data and then replaced the Shakespeare_char demo to train a char sorting model.

import itertools
import random

# Set the number of samples you want
num_samples = 10000

# Generate the dataset
data = []
for _ in range(num_samples):
    # Create a random string using ABC characters
    string = ''.join(random.choice('ABC') for _ in range(random.randint(1, 5)))
    # Sort the string to get the correct answer
    sorted_string = ''.join(sorted(string))
    # Add the unsorted and sorted pair to the dataset
    data.append(f"{string}|{sorted_string}")

# Save the dataset to a file
with open('sorting_dataset.txt', 'w') as f:
    for line in data:
        f.write(line + "\n")

like this:

[Nuclear6]

@gitzhangzhao
This is what I want. The training data format is quite different from the Shakespeare character set. How did you modify the training script? It looks like the vertical lines | are also predicted.