From 5b5ee66cbe1293b8820a215310ea88a6dac11e67 Mon Sep 17 00:00:00 2001 From: thiagokokada Date: Tue, 3 Sep 2024 13:44:41 +0000 Subject: [PATCH] README/rss: update --- rss.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rss.xml b/rss.xml index 94c64fc..dc612d0 100644 --- a/rss.xml +++ b/rss.xml @@ -6,7 +6,7 @@ My coding experience with LLM https://github.com/thiagokokada/blog/blob/main/posts/2024-09-02/01-my-coding-experience-with-llm.md - <p>I don't want to talk about AI. AI fatigue is real, with (garbage) AI content everywhere you go. However, I like to talk about tools, and AIs, or more specifically, <a href="https://en.wikipedia.org/wiki/Large_language_model">Large Language Models</a> (LLM), can be really useful tools. Maybe someday we will have our AI overlords dominating everything (including my job), but right now I don't feel particularly threatened given my experience with AIs.</p> <p>Given that <a href="https://mashable.com/article/google-search-low-quality-research">Google Search is getting worse</a>, I am slowly relying in LLMs more and more. This is the focus of this post, basically <a href="https://www.rottentomatoes.com/m/the_good_the_bad_and_the_ugly">the good, the bad and the ugly</a> (by the way, never saw this film) of LLMs, especially focused for solving coding problems. I will take a look at the my chat history in <a href="https://chatgpt.com/">ChatGPT</a> and see where it was a good tool, where it shined, and where it just completely wasted my time.</p> <p>A disclaimer, I never bothered to buy a ChatGPT subscription because I don't feel that the tool is useful enough (at least yet), so most of the examples here are either with <a href="https://openai.com/index/chatgpt/">GPT-3.5</a> or <a href="https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/">GPT-4o mini</a>, since those where the models offered by free in ChatGPT.</p> <p>An addendum, I have some experience with <a href="https://github.com/features/copilot">GitHub Copilot</a> (another LLM, but this one runs inside your IDE or favorite text editor) in <code>$PREVIOUS_JOB</code>. But my experience with it is even more underwhelming than what I am going to describe in this post: it was basically really good for generating fixture for tests and between ok-ish to plain bad for everything else. Yes, that's would fit in a tweet (or xeet?).</p> <h2>The good</h2> <p><a href="https://chatgpt.com/share/c58a1432-2761-4954-acac-cade29942c34">Chat history</a>.</p> <blockquote> <p>Is there any way that I can pass a context for a net.conn to cancel its Read()?</p> </blockquote> <p>This is one of the good cases where ChatGPT felt to me that I was talking with an expert in Go. To start, I didn't even mention in my initial prompt that I was talking about Go, but it correctly inferred so. Also, the solution overview was great:</p> <blockquote> <ol> <li><strong>Use a separate goroutine</strong> to handle the context cancellation.</li> <li><strong>Set a deadline</strong> on the connection using <code>SetReadDeadline()</code> to unblock the <code>Read()</code> when the context is canceled.</li> </ol> </blockquote> <p>Before asking ChatGPT about this issue I had already coded my own naive solution that was something similar to this:</p> <pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">foo</span>(<span style="color:#a6e22e">ctx</span> <span style="color:#a6e22e">context</span>.<span style="color:#a6e22e">Context</span>) { </span></span><span style="display:flex;"><span> <span style="color:#75715e">// ... </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#a6e22e">done</span> <span style="color:#f92672">:=</span> make(<span style="color:#66d9ef">chan</span> <span style="color:#66d9ef">struct</span>{}) </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">var</span> <span style="color:#a6e22e">n</span> <span style="color:#66d9ef">int</span> </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">var</span> <span style="color:#a6e22e">err</span> <span style="color:#66d9ef">error</span> </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">go</span> <span style="color:#66d9ef">func</span>() { </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span> = <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">Read</span>(<span style="color:#a6e22e">buf</span>) </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">done</span> <span style="color:#f92672">&lt;-</span><span style="color:#66d9ef">struct</span>{}{} </span></span><span style="display:flex;"><span> }() </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">select</span> { </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Done</span>(): </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">Close</span>() </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#ae81ff">0</span>, <span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Err</span>() </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span>: </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span> </span></span><span style="display:flex;"><span> } </span></span><span style="display:flex;"><span> <span style="color:#75715e">// ... </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>} </span></span></code></pre><p>Compared this solution with the one from ChatGPT:</p> <pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">readWithContext</span>(<span style="color:#a6e22e">ctx</span> <span style="color:#a6e22e">context</span>.<span style="color:#a6e22e">Context</span>, <span style="color:#a6e22e">conn</span> <span style="color:#a6e22e">net</span>.<span style="color:#a6e22e">Conn</span>, <span style="color:#a6e22e">buf</span> []<span style="color:#66d9ef">byte</span>) (<span style="color:#66d9ef">int</span>, <span style="color:#66d9ef">error</span>) { </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">done</span> <span style="color:#f92672">:=</span> make(<span style="color:#66d9ef">chan</span> <span style="color:#66d9ef">struct</span>{}) </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">var</span> <span style="color:#a6e22e">n</span> <span style="color:#66d9ef">int</span> </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">var</span> <span style="color:#a6e22e">err</span> <span style="color:#66d9ef">error</span> </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span> <span style="color:#75715e">// Start a goroutine to perform the read </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">go</span> <span style="color:#66d9ef">func</span>() { </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span> = <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">Read</span>(<span style="color:#a6e22e">buf</span>) </span></span><span style="display:flex;"><span> close(<span style="color:#a6e22e">done</span>) </span></span><span style="display:flex;"><span> }() </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">select</span> { </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Done</span>(): <span style="color:#75715e">// Context was canceled or timed out </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#75715e">// Set a short deadline to unblock the Read() </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">SetReadDeadline</span>(<span style="color:#a6e22e">time</span>.<span style="color:#a6e22e">Now</span>()) </span></span><span style="display:flex;"><span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span> <span style="color:#75715e">// Wait for the read to finish </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">return</span> <span style="color:#ae81ff">0</span>, <span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Err</span>() </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span>: <span style="color:#75715e">// Read finished successfully </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span> </span></span><span style="display:flex;"><span> } </span></span><span style="display:flex;"><span>} </span></span></code></pre><p>The ChatGPT version is much better because:</p> <ol> <li>It doesn't close the connection socket allowing it to be reused</li> <li>It waits until the goroutine is finished to avoid leaks</li> <li>It closes the channel after it finishes</li> </ol> <p>Now, I don't claim to be a Go expert (far from it), but this was one of those cases where I felt that I was being guided by someone that is coding in Go for years.</p> <p>But it didn't end there, I asked:</p> <blockquote> <p>If I want to reuse the connection, do I need to reset the SetReadDeadline?</p> </blockquote> <p>Since in this particular case I wanted the underlying connection to be reused, and ChatGPT correctly amended the code:</p> <pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">readWithContext</span>(<span style="color:#a6e22e">ctx</span> <span style="color:#a6e22e">context</span>.<span style="color:#a6e22e">Context</span>, <span style="color:#a6e22e">conn</span> <span style="color:#a6e22e">net</span>.<span style="color:#a6e22e">Conn</span>, <span style="color:#a6e22e">buf</span> []<span style="color:#66d9ef">byte</span>) (<span style="color:#66d9ef">int</span>, <span style="color:#66d9ef">error</span>) { </span></span><span style="display:flex;"><span> <span style="color:#75715e">// Same as before </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">select</span> { </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Done</span>(): <span style="color:#75715e">// Context was canceled or timed out </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#75715e">// Set a short deadline to unblock the Read() </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">SetReadDeadline</span>(<span style="color:#a6e22e">time</span>.<span style="color:#a6e22e">Now</span>()) </span></span><span style="display:flex;"><span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span> <span style="color:#75715e">// Wait for the read to finish </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> </span></span><span style="display:flex;"><span> <span style="color:#75715e">// Reset the deadline so the connection can be reused </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">SetReadDeadline</span>(<span style="color:#a6e22e">time</span>.<span style="color:#a6e22e">Time</span>{}) </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#ae81ff">0</span>, <span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Err</span>() </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span>: <span style="color:#75715e">// Read finished successfully </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span> </span></span><span style="display:flex;"><span> } </span></span><span style="display:flex;"><span>} </span></span></code></pre><p>Also the explanation on why this happens was good:</p> <blockquote> <p>When you set a read deadline with <code>conn.SetReadDeadline(time.Now())</code> to unblock the <code>Read()</code> call, this deadline remains in effect for future <code>Read()</code> operations. This means that if you attempt another read operation after this, it could immediately fail with a timeout error if the deadline is still in effect. To reuse the connection, you'll need to remove the deadline or set a new one. You can do this by calling <code>conn.SetReadDeadline(time.Time{})</code>, which clears any existing deadline.</p> </blockquote> <p>I had one last question that I kind already knew the answer:</p> <blockquote> <p>Do we need to defer the clearing?</p> </blockquote> <p>And ChatGPT correctly returned:</p> <blockquote> <p>Whether you need to defer the clearing of the read deadline depends on how your code is structured. If you want to ensure that the read deadline is always cleared after a context cancellation, even if the code panics or exits early, then deferring the <code>conn.SetReadDeadline(time.Time{})</code> call is a good idea.</p> </blockquote> <p>This whole interaction was one of the best ones I ever had in ChatGPT, one that it gave me an actual functional code with good explanation on what was happening. But still, it didn't talked at all about the gotcha of setting the read deadline and reusing the underlying connection, needing another prompt to give me this information.</p> <p>Also, there is one issue in ChatGPT solution though: we are discarding the values of the buffer once the context is done by returning <code>n = 0</code> instead of the number of bytes read. In this particular case this is what I wanted, but this is a possible issue that ChatGPT didn't talk.</p> <p>For comparison, <a href="https://stackoverflow.com/questions/61637359/how-to-use-context-context-with-tcp-connection-read">this</a> is the best that I got searching in both DuckDuckGo and Google, that is where I kind based my first solution on. I still not sure where ChatGPT could give me such a good answer for this question, because I just couldn't find anything remotely similar during my searches.</p> <h2>The amazing</h2> <p><a href="https://chatgpt.com/share/0c9a148f-4cfd-4b4e-aa84-7486e185125b">Chat history</a>.</p> <blockquote> <p>I want to write an extension to <a href="https://github.com/yuin/goldmark/">https://github.com/yuin/goldmark/</a> that allows me to rewrite <code>[link](../2024-07-30/post.md)</code> to <code>[link](https://kokada.capivaras.dev/slug)</code></p> </blockquote> <p>Here is the resulting code:</p> <pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#f92672">package</span> <span style="color:#a6e22e">linkrewriter</span> </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#f92672">import</span> ( </span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;fmt&#34;</span> </span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;path&#34;</span> </span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;strings&#34;</span> </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;github.com/yuin/goldmark&#34;</span> </span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;github.com/yuin/goldmark/ast&#34;</span> </span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;github.com/yuin/goldmark/parser&#34;</span> </span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;github.com/yuin/goldmark/renderer&#34;</span> </span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;github.com/yuin/goldmark/util&#34;</span> </span></span><span style="display:flex;"><span>) </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e">// LinkRewriter is the main struct for your extension </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">type</span> <span style="color:#a6e22e">LinkRewriter</span> <span style="color:#66d9ef">struct</span>{} </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e">// NewLinkRewriter returns a new instance of LinkRewriter </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">NewLinkRewriter</span>() <span style="color:#f92672">*</span><span style="color:#a6e22e">LinkRewriter</span> { </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#f92672">&amp;</span><span style="color:#a6e22e">LinkRewriter</span>{} </span></span><span style="display:flex;"><span>} </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e">// Extend will be called by Goldmark to add your extension </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> (<span style="color:#a6e22e">e</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">LinkRewriter</span>) <span style="color:#a6e22e">Extend</span>(<span style="color:#a6e22e">m</span> <span style="color:#a6e22e">goldmark</span>.<span style="color:#a6e22e">Markdown</span>) { </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">m</span>.<span style="color:#a6e22e">Parser</span>().<span style="color:#a6e22e">AddOptions</span>(<span style="color:#a6e22e">parser</span>.<span style="color:#a6e22e">WithASTTransformers</span>(<span style="color:#a6e22e">util</span>.<span style="color:#a6e22e">Prioritized</span>(<span style="color:#a6e22e">e</span>, <span style="color:#ae81ff">0</span>))) </span></span><span style="display:flex;"><span>} </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e">// Transform is the method that modifies the AST </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> (<span style="color:#a6e22e">e</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">LinkRewriter</span>) <span style="color:#a6e22e">Transform</span>(<span style="color:#a6e22e">node</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">Document</span>, <span style="color:#a6e22e">reader</span> <span style="color:#a6e22e">text</span>.<span style="color:#a6e22e">Reader</span>, <span style="color:#a6e22e">pc</span> <span style="color:#a6e22e">parser</span>.<span style="color:#a6e22e">Context</span>) { </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">Walk</span>(<span style="color:#a6e22e">node</span>, <span style="color:#66d9ef">func</span>(<span style="color:#a6e22e">n</span> <span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">Node</span>, <span style="color:#a6e22e">entering</span> <span style="color:#66d9ef">bool</span>) (<span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">WalkStatus</span>, <span style="color:#66d9ef">error</span>) { </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> !<span style="color:#a6e22e">entering</span> { </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">WalkContinue</span>, <span style="color:#66d9ef">nil</span> </span></span><span style="display:flex;"><span> } </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> <span style="color:#a6e22e">link</span>, <span style="color:#a6e22e">ok</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">n</span>.(<span style="color:#f92672">*</span><span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">Link</span>); <span style="color:#a6e22e">ok</span> { </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">rewriteLink</span>(<span style="color:#a6e22e">link</span>) </span></span><span style="display:flex;"><span> } </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">WalkContinue</span>, <span style="color:#66d9ef">nil</span> </span></span><span style="display:flex;"><span> }) </span></span><span style="display:flex;"><span>} </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e">// rewriteLink modifies the link URL </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">rewriteLink</span>(<span style="color:#a6e22e">link</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">Link</span>) { </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">url</span> <span style="color:#f92672">:=</span> string(<span style="color:#a6e22e">link</span>.<span style="color:#a6e22e">Destination</span>) </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> <span style="color:#a6e22e">strings</span>.<span style="color:#a6e22e">HasPrefix</span>(<span style="color:#a6e22e">url</span>, <span style="color:#e6db74">&#34;../&#34;</span>) { </span></span><span style="display:flex;"><span> <span style="color:#75715e">// Modify this part according to your slug generation logic </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#a6e22e">slug</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">generateSlug</span>(<span style="color:#a6e22e">url</span>) </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">link</span>.<span style="color:#a6e22e">Destination</span> = []byte(<span style="color:#a6e22e">fmt</span>.<span style="color:#a6e22e">Sprintf</span>(<span style="color:#e6db74">&#34;https://kokada.capivaras.dev/%s&#34;</span>, <span style="color:#a6e22e">slug</span>)) </span></span><span style="display:flex;"><span> } </span></span><span style="display:flex;"><span>} </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e">// generateSlug generates a slug from the given URL </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">generateSlug</span>(<span style="color:#a6e22e">url</span> <span style="color:#66d9ef">string</span>) <span style="color:#66d9ef">string</span> { </span></span><span style="display:flex;"><span> <span style="color:#75715e">// Assuming the slug is the last part of the URL without the extension </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">strings</span>.<span style="color:#a6e22e">TrimSuffix</span>(<span style="color:#a6e22e">path</span>.<span style="color:#a6e22e">Base</span>(<span style="color:#a6e22e">url</span>), <span style="color:#a6e22e">path</span>.<span style="color:#a6e22e">Ext</span>(<span style="color:#a6e22e">url</span>)) </span></span><span style="display:flex;"><span>} </span></span></code></pre><p>This one will need some context. <a href="https://github.com/yuin/goldmark/">Goldmark</a>, the Markdown renderer that <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-08-24/01-making-a-blog-for-the-next-10-years.md">powers this blog</a>, does not have good examples or documentation (at least I didn't found neither).</p> <p>So I was not really hopeful I would get any answers in ChatGPT, but not only it put me in the correct path, the code was almost correct. It did had some unused and missing imports, and there were some compilation errors that were easily fixed. And when it registered the extension, it also added a <code>html.WithHardWraps()</code> renderer for no reason at all.</p> <p>But this is one of the experiences in LLM that most impressed me. I had no idea at all on how to solve this problem, and ChatGPT basically gave me the solution. A <a href="https://github.com/thiagokokada/blog/blob/3a2ea58a0638d1005617d25eabeae0626933db46/link_rewriter.go">modified version of the code</a> from this chat is still used to power this blog to this day, that also demonstrates that the code is not garbage (since now I understand what it does and I can even extend it).</p> <p>I am still not sure how the license situation of this code is though. Does licenses even make sense in AI generated code?</p> <h2>The bad</h2> <p><a href="https://chatgpt.com/share/95fcd777-b03c-4cfe-8d1d-12e53f5a4dfe">Chat history</a>.</p> <blockquote> <p>I want to configure VM options inside IntelliJ by project</p> </blockquote> <blockquote> <p>My project uses sbt, and I want to set this in a way that I can share with my colleagues</p> </blockquote> <blockquote> <p>I am using a build.sbt file, but IntelliJ is not loading it's JVM options and I need to set this up manually. I want to automate this</p> </blockquote> <p>This is an example of my most common interaction with ChatGPT: it generally gives some options that could work, but not in my specific case. So I start to ask for more information until it eventually starts to <a href="https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence)">hallucinate</a>.</p> <p>In this particular case, the hallucination was in:</p> <blockquote> <p>The sbt-idea-plugin can generate IntelliJ IDEA project files based on your SBT configuration, including JVM options. This allows you to ensure that IntelliJ loads the correct settings from the SBT configuration.</p> <p>Steps:</p> <ol> <li>Add the <code>sbt-idea-plugin</code> to your <code>project/plugins.sbt</code> file:</li> </ol> <pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>addSbtPlugin<span style="color:#f92672">(</span><span style="color:#e6db74">&#34;com.github.mwz&#34;</span> <span style="color:#f92672">%</span> <span style="color:#e6db74">&#34;sbt-idea&#34;</span> <span style="color:#f92672">%</span> <span style="color:#e6db74">&#34;2.4.0&#34;</span><span style="color:#f92672">)</span> </span></span></code></pre><ol start="2"> <li>Run the following command in your terminal to regenerate IntelliJ IDEA project files:</li> </ol> <pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>sbt gen-idea </span></span></code></pre></blockquote> <p>Yes, there is a <a href="https://github.com/JetBrains/sbt-idea-plugin"><code>sbt-idea-plugin</code></a>, but no, it doesn't generate project files based in my SBT configuration. It is not even from the repository that it is supposed to be pointing too.</p> <p>I will just not say that this interaction was a waste of time because some of the other ideas can be applied, but this was the kind of case where I ended up just being disappointed because the thing that seemed to be a possible solution for my problem ended up not existing at all.</p> <h2>The ugly</h2> <p>There are lots of examples where ChatGPT just plain disappointed me. Here is a few:</p> <blockquote> <p>What is the language that is a fork of Go that has extra features like map?</p> </blockquote> <p><a href="https://chatgpt.com/share/2b854bbd-d376-4049-97fe-b47321c1ef07">Chat history</a>.</p> <p>This is one of those cases where I saw something once and was trying to remember. ChatGPT just gave me lots of answers that were not a fork of Go (<a href="https://vlang.io/">V</a>, <a href="https://github.com/gluon-lang/gluon">Gluon</a>, <a href="https://www.ponylang.io">Pony</a>), hallucinated a programming language (Gomini), and said that <a href="https://ziglang.org/">Zig</a> is focused in STEM.</p> <p>By the way, I eventually remember what the language was: <a href="https://goplus.org/">Go+</a>. To be fair with ChatGPT, I misremembered that the Go+ had map (as the <a href="https://en.wikipedia.org/wiki/Map_(higher-order_function)">high-order function</a>, not a hashmap that both Go and Go+ have). But still, Go+ explicit announces that it is a language for STEM everywhere in their website, and even them ChatGPT couldn't answer.</p> <blockquote> <p>In IntelliJ, I need to pass some VM options to the run configuration for my application to run tests, e.g.:</p> <pre><code>--add-exports=java.base/sun.security.x509=ALL-UNNAMED --add-opens=java.base/sun.security.ssl=ALL-UNNAMED -Dconfig.resource=api.test.conf </code></pre> <p>However, the <code>api.test.conf</code> may be <code>jobs.api.conf</code> or something else depending on the subproject I am running inside Idea. Can I parametrize this in some way?</p> </blockquote> <p><a href="https://chatgpt.com/share/aa9bd051-65de-49d9-8d56-0e28de94722c">Chat history</a></p> <p>So quick answer: you can't. IntelliJ macros are not powerful enough for this. ChatGPT itself said so when I asked about the IntelliJ macros <a href="https://chatgpt.com/share/7a057bf5-bcec-458b-9432-98e3f551ad28">directly</a>.</p> <p>But in this particular case, ChatGPT happily said &quot;yes, you can parameterize VM options in IntelliJ IDEA to make them more flexible depending on the subproject or other context&quot;. Thanks for wasting my time, I guess.</p> <h2>Conclusion</h2> <p>I do think that LLMs, in particular ChatGPT since this is the one I have more experience with, are improving. But still, if there is anything that I concluded after putting all those experiences in a post is that you still need someone that knows at least as much about whatever you're asking ChatGPT to actually validate the answer.</p> <p>Not only because LLMs hallucinates a lot, but also because even in the &quot;good&quot; example the LLM needed extra input to figure out about the possible drawback of the solution. The only case where I felt that ChatGPT outsmarted me was the &quot;amazing&quot; case, but this was literally the only example that I could find in all my chat history with ChatGPT that was like this.</p> <p>So yes, at least for now I still don't think AI will be our overlords. Maybe someday.</p> + <p>I don't want to talk about AI. AI fatigue is real, with (garbage) AI content everywhere you go. However, I like to talk about tools, and AIs, or more specifically, <a href="https://en.wikipedia.org/wiki/Large_language_model">Large Language Models</a> (LLM), can be really useful tools. Maybe someday we will have our AI overlords dominating everything (including my job), but right now I don't feel particularly threatened given my experience with AIs.</p> <p>Given that <a href="https://mashable.com/article/google-search-low-quality-research">Google Search is getting worse</a>, I am slowly relying in LLMs more and more. This is the focus of this post, basically <a href="https://www.rottentomatoes.com/m/the_good_the_bad_and_the_ugly">the good, the bad and the ugly</a> (by the way, never saw this film) of LLMs, especially focused for solving coding problems. I will take a look at the my chat history in <a href="https://chatgpt.com/">ChatGPT</a> and see where it was a good tool, where it shined, and where it just completely wasted my time.</p> <p>A disclaimer, I never bothered to buy a ChatGPT subscription because I don't feel that the tool is useful enough (at least yet), so most of the examples here are either with <a href="https://openai.com/index/chatgpt/">GPT-3.5</a> or <a href="https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/">GPT-4o mini</a>, since those where the models offered by free in ChatGPT.</p> <p>An addendum, I have some experience with <a href="https://github.com/features/copilot">GitHub Copilot</a> (another LLM, but this one runs inside your IDE or favorite text editor) in <code>$PREVIOUS_JOB</code>. But my experience with it is even more underwhelming than what I am going to describe in this post: it was basically really good for generating fixture for tests and between ok-ish to plain bad for everything else. Yes, that's would fit in a tweet (or xeet?).</p> <h2>The good</h2> <p><a href="https://chatgpt.com/share/c58a1432-2761-4954-acac-cade29942c34">Chat history</a>.</p> <blockquote> <p>Is there any way that I can pass a context for a net.conn to cancel its Read()?</p> </blockquote> <p>This is one of the good cases where ChatGPT felt to me that I was talking with an expert in Go. To start, I didn't even mention in my initial prompt that I was talking about Go, but it correctly inferred so. Also, the solution overview was great:</p> <blockquote> <ol> <li><strong>Use a separate goroutine</strong> to handle the context cancellation.</li> <li><strong>Set a deadline</strong> on the connection using <code>SetReadDeadline()</code> to unblock the <code>Read()</code> when the context is canceled.</li> </ol> </blockquote> <p>Before asking ChatGPT about this issue I had already coded my own naive solution that was something similar to this:</p> <pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">foo</span>(<span style="color:#a6e22e">ctx</span> <span style="color:#a6e22e">context</span>.<span style="color:#a6e22e">Context</span>) { </span></span><span style="display:flex;"><span> <span style="color:#75715e">// ... </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#a6e22e">done</span> <span style="color:#f92672">:=</span> make(<span style="color:#66d9ef">chan</span> <span style="color:#66d9ef">struct</span>{}) </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">var</span> <span style="color:#a6e22e">n</span> <span style="color:#66d9ef">int</span> </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">var</span> <span style="color:#a6e22e">err</span> <span style="color:#66d9ef">error</span> </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">go</span> <span style="color:#66d9ef">func</span>() { </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span> = <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">Read</span>(<span style="color:#a6e22e">buf</span>) </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">done</span> <span style="color:#f92672">&lt;-</span><span style="color:#66d9ef">struct</span>{}{} </span></span><span style="display:flex;"><span> }() </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">select</span> { </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Done</span>(): </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">Close</span>() </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#ae81ff">0</span>, <span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Err</span>() </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span>: </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span> </span></span><span style="display:flex;"><span> } </span></span><span style="display:flex;"><span> <span style="color:#75715e">// ... </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>} </span></span></code></pre><p>Compared this solution with the one from ChatGPT:</p> <pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">readWithContext</span>(<span style="color:#a6e22e">ctx</span> <span style="color:#a6e22e">context</span>.<span style="color:#a6e22e">Context</span>, <span style="color:#a6e22e">conn</span> <span style="color:#a6e22e">net</span>.<span style="color:#a6e22e">Conn</span>, <span style="color:#a6e22e">buf</span> []<span style="color:#66d9ef">byte</span>) (<span style="color:#66d9ef">int</span>, <span style="color:#66d9ef">error</span>) { </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">done</span> <span style="color:#f92672">:=</span> make(<span style="color:#66d9ef">chan</span> <span style="color:#66d9ef">struct</span>{}) </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">var</span> <span style="color:#a6e22e">n</span> <span style="color:#66d9ef">int</span> </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">var</span> <span style="color:#a6e22e">err</span> <span style="color:#66d9ef">error</span> </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span> <span style="color:#75715e">// Start a goroutine to perform the read </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">go</span> <span style="color:#66d9ef">func</span>() { </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span> = <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">Read</span>(<span style="color:#a6e22e">buf</span>) </span></span><span style="display:flex;"><span> close(<span style="color:#a6e22e">done</span>) </span></span><span style="display:flex;"><span> }() </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">select</span> { </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Done</span>(): <span style="color:#75715e">// Context was canceled or timed out </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#75715e">// Set a short deadline to unblock the Read() </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">SetReadDeadline</span>(<span style="color:#a6e22e">time</span>.<span style="color:#a6e22e">Now</span>()) </span></span><span style="display:flex;"><span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span> <span style="color:#75715e">// Wait for the read to finish </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">return</span> <span style="color:#ae81ff">0</span>, <span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Err</span>() </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span>: <span style="color:#75715e">// Read finished successfully </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span> </span></span><span style="display:flex;"><span> } </span></span><span style="display:flex;"><span>} </span></span></code></pre><p>The ChatGPT version is much better because:</p> <ol> <li>It doesn't close the connection socket allowing it to be reused</li> <li>It waits until the goroutine is finished to avoid leaks</li> <li>It closes the channel after it finishes</li> </ol> <p>Now, I don't claim to be a Go expert (far from it), but this was one of those cases where I felt that I was being guided by someone that is coding in Go for years.</p> <p>But it didn't end there, I asked:</p> <blockquote> <p>If I want to reuse the connection, do I need to reset the SetReadDeadline?</p> </blockquote> <p>Since in this particular case I wanted the underlying connection to be reused, and ChatGPT correctly amended the code:</p> <pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">readWithContext</span>(<span style="color:#a6e22e">ctx</span> <span style="color:#a6e22e">context</span>.<span style="color:#a6e22e">Context</span>, <span style="color:#a6e22e">conn</span> <span style="color:#a6e22e">net</span>.<span style="color:#a6e22e">Conn</span>, <span style="color:#a6e22e">buf</span> []<span style="color:#66d9ef">byte</span>) (<span style="color:#66d9ef">int</span>, <span style="color:#66d9ef">error</span>) { </span></span><span style="display:flex;"><span> <span style="color:#75715e">// Same as before </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">select</span> { </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Done</span>(): <span style="color:#75715e">// Context was canceled or timed out </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#75715e">// Set a short deadline to unblock the Read() </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">SetReadDeadline</span>(<span style="color:#a6e22e">time</span>.<span style="color:#a6e22e">Now</span>()) </span></span><span style="display:flex;"><span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span> <span style="color:#75715e">// Wait for the read to finish </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> </span></span><span style="display:flex;"><span> <span style="color:#75715e">// Reset the deadline so the connection can be reused </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">SetReadDeadline</span>(<span style="color:#a6e22e">time</span>.<span style="color:#a6e22e">Time</span>{}) </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#ae81ff">0</span>, <span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Err</span>() </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span>: <span style="color:#75715e">// Read finished successfully </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span> </span></span><span style="display:flex;"><span> } </span></span><span style="display:flex;"><span>} </span></span></code></pre><p>Also the explanation on why this happens was good:</p> <blockquote> <p>When you set a read deadline with <code>conn.SetReadDeadline(time.Now())</code> to unblock the <code>Read()</code> call, this deadline remains in effect for future <code>Read()</code> operations. This means that if you attempt another read operation after this, it could immediately fail with a timeout error if the deadline is still in effect. To reuse the connection, you'll need to remove the deadline or set a new one. You can do this by calling <code>conn.SetReadDeadline(time.Time{})</code>, which clears any existing deadline.</p> </blockquote> <p>I had one last question that I kind already knew the answer:</p> <blockquote> <p>Do we need to defer the clearing?</p> </blockquote> <p>And ChatGPT correctly returned:</p> <blockquote> <p>Whether you need to defer the clearing of the read deadline depends on how your code is structured. If you want to ensure that the read deadline is always cleared after a context cancellation, even if the code panics or exits early, then deferring the <code>conn.SetReadDeadline(time.Time{})</code> call is a good idea.</p> </blockquote> <p>This whole interaction was one of the best ones I ever had in ChatGPT, one that it gave me an actual functional code with good explanation on what was happening. But still, it didn't talked at all about the gotcha of setting the read deadline and reusing the underlying connection, needing another prompt to give me this information.</p> <p>Also, there is one issue in ChatGPT solution though: we are discarding the values of the buffer once the context is done by returning <code>n = 0</code> instead of the number of bytes read. In this particular case this is what I wanted, but this is a possible issue that ChatGPT didn't talk.</p> <p>For comparison, <a href="https://stackoverflow.com/questions/61637359/how-to-use-context-context-with-tcp-connection-read">this</a> is the best that I got searching in both DuckDuckGo and Google, that is where I kind based my first solution on. I still not sure where ChatGPT could give me such a good answer for this question, because I just couldn't find anything remotely similar during my searches.</p> <h2>The amazing</h2> <p><a href="https://chatgpt.com/share/0c9a148f-4cfd-4b4e-aa84-7486e185125b">Chat history</a>.</p> <blockquote> <p>I want to write an extension to <a href="https://github.com/yuin/goldmark/">https://github.com/yuin/goldmark/</a> that allows me to rewrite <code>[link](../2024-07-30/post.md)</code> to <code>[link](https://kokada.capivaras.dev/slug)</code></p> </blockquote> <p>Here is the generated code:</p> <pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#f92672">package</span> <span style="color:#a6e22e">linkrewriter</span> </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#f92672">import</span> ( </span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;fmt&#34;</span> </span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;path&#34;</span> </span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;strings&#34;</span> </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;github.com/yuin/goldmark&#34;</span> </span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;github.com/yuin/goldmark/ast&#34;</span> </span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;github.com/yuin/goldmark/parser&#34;</span> </span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;github.com/yuin/goldmark/renderer&#34;</span> </span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;github.com/yuin/goldmark/util&#34;</span> </span></span><span style="display:flex;"><span>) </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e">// LinkRewriter is the main struct for your extension </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">type</span> <span style="color:#a6e22e">LinkRewriter</span> <span style="color:#66d9ef">struct</span>{} </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e">// NewLinkRewriter returns a new instance of LinkRewriter </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">NewLinkRewriter</span>() <span style="color:#f92672">*</span><span style="color:#a6e22e">LinkRewriter</span> { </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#f92672">&amp;</span><span style="color:#a6e22e">LinkRewriter</span>{} </span></span><span style="display:flex;"><span>} </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e">// Extend will be called by Goldmark to add your extension </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> (<span style="color:#a6e22e">e</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">LinkRewriter</span>) <span style="color:#a6e22e">Extend</span>(<span style="color:#a6e22e">m</span> <span style="color:#a6e22e">goldmark</span>.<span style="color:#a6e22e">Markdown</span>) { </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">m</span>.<span style="color:#a6e22e">Parser</span>().<span style="color:#a6e22e">AddOptions</span>(<span style="color:#a6e22e">parser</span>.<span style="color:#a6e22e">WithASTTransformers</span>(<span style="color:#a6e22e">util</span>.<span style="color:#a6e22e">Prioritized</span>(<span style="color:#a6e22e">e</span>, <span style="color:#ae81ff">0</span>))) </span></span><span style="display:flex;"><span>} </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e">// Transform is the method that modifies the AST </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> (<span style="color:#a6e22e">e</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">LinkRewriter</span>) <span style="color:#a6e22e">Transform</span>(<span style="color:#a6e22e">node</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">Document</span>, <span style="color:#a6e22e">reader</span> <span style="color:#a6e22e">text</span>.<span style="color:#a6e22e">Reader</span>, <span style="color:#a6e22e">pc</span> <span style="color:#a6e22e">parser</span>.<span style="color:#a6e22e">Context</span>) { </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">Walk</span>(<span style="color:#a6e22e">node</span>, <span style="color:#66d9ef">func</span>(<span style="color:#a6e22e">n</span> <span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">Node</span>, <span style="color:#a6e22e">entering</span> <span style="color:#66d9ef">bool</span>) (<span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">WalkStatus</span>, <span style="color:#66d9ef">error</span>) { </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> !<span style="color:#a6e22e">entering</span> { </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">WalkContinue</span>, <span style="color:#66d9ef">nil</span> </span></span><span style="display:flex;"><span> } </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> <span style="color:#a6e22e">link</span>, <span style="color:#a6e22e">ok</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">n</span>.(<span style="color:#f92672">*</span><span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">Link</span>); <span style="color:#a6e22e">ok</span> { </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">rewriteLink</span>(<span style="color:#a6e22e">link</span>) </span></span><span style="display:flex;"><span> } </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">WalkContinue</span>, <span style="color:#66d9ef">nil</span> </span></span><span style="display:flex;"><span> }) </span></span><span style="display:flex;"><span>} </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e">// rewriteLink modifies the link URL </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">rewriteLink</span>(<span style="color:#a6e22e">link</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">Link</span>) { </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">url</span> <span style="color:#f92672">:=</span> string(<span style="color:#a6e22e">link</span>.<span style="color:#a6e22e">Destination</span>) </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> <span style="color:#a6e22e">strings</span>.<span style="color:#a6e22e">HasPrefix</span>(<span style="color:#a6e22e">url</span>, <span style="color:#e6db74">&#34;../&#34;</span>) { </span></span><span style="display:flex;"><span> <span style="color:#75715e">// Modify this part according to your slug generation logic </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#a6e22e">slug</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">generateSlug</span>(<span style="color:#a6e22e">url</span>) </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">link</span>.<span style="color:#a6e22e">Destination</span> = []byte(<span style="color:#a6e22e">fmt</span>.<span style="color:#a6e22e">Sprintf</span>(<span style="color:#e6db74">&#34;https://kokada.capivaras.dev/%s&#34;</span>, <span style="color:#a6e22e">slug</span>)) </span></span><span style="display:flex;"><span> } </span></span><span style="display:flex;"><span>} </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e">// generateSlug generates a slug from the given URL </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">generateSlug</span>(<span style="color:#a6e22e">url</span> <span style="color:#66d9ef">string</span>) <span style="color:#66d9ef">string</span> { </span></span><span style="display:flex;"><span> <span style="color:#75715e">// Assuming the slug is the last part of the URL without the extension </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">strings</span>.<span style="color:#a6e22e">TrimSuffix</span>(<span style="color:#a6e22e">path</span>.<span style="color:#a6e22e">Base</span>(<span style="color:#a6e22e">url</span>), <span style="color:#a6e22e">path</span>.<span style="color:#a6e22e">Ext</span>(<span style="color:#a6e22e">url</span>)) </span></span><span style="display:flex;"><span>} </span></span></code></pre><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#f92672">package</span> <span style="color:#a6e22e">main</span> </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#f92672">import</span> ( </span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;github.com/yuin/goldmark&#34;</span> </span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;github.com/yuin/goldmark/renderer/html&#34;</span> </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">linkrewriter</span> <span style="color:#e6db74">&#34;path/to/your/linkrewriter&#34;</span> </span></span><span style="display:flex;"><span>) </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">main</span>() { </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">md</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">goldmark</span>.<span style="color:#a6e22e">New</span>( </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">goldmark</span>.<span style="color:#a6e22e">WithExtensions</span>( </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">linkrewriter</span>.<span style="color:#a6e22e">NewLinkRewriter</span>(), </span></span><span style="display:flex;"><span> ), </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">goldmark</span>.<span style="color:#a6e22e">WithRendererOptions</span>( </span></span><span style="display:flex;"><span> <span style="color:#a6e22e">html</span>.<span style="color:#a6e22e">WithHardWraps</span>(), </span></span><span style="display:flex;"><span> ), </span></span><span style="display:flex;"><span> ) </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span> <span style="color:#75715e">// Use `md.Convert` to convert your markdown to HTML as usual </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>} </span></span></code></pre><p>This one will need some context. <a href="https://github.com/yuin/goldmark/">Goldmark</a>, the Markdown renderer that <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-08-24/01-making-a-blog-for-the-next-10-years.md">powers this blog</a>, does not have good examples or documentation (at least I didn't found neither).</p> <p>So I was not really hopeful I would get any answers in ChatGPT, but not only it put me in the correct path, the code was almost correct. It did had some unused and missing imports, and there were some compilation errors that were easily fixed. And when it registered the extension, it also added a <code>html.WithHardWraps()</code> renderer for no reason at all.</p> <p>But this is one of the experiences in LLM that most impressed me. I had no idea at all on how to solve this problem, and ChatGPT basically gave me the solution. A <a href="https://github.com/thiagokokada/blog/blob/3a2ea58a0638d1005617d25eabeae0626933db46/link_rewriter.go">modified version of the code</a> from this chat is still used to power this blog to this day, that also demonstrates that the code is not garbage (since now I understand what it does and I can even extend it).</p> <p>I am still not sure how the license situation of this code is though. Does licenses even make sense in AI generated code?</p> <h2>The bad</h2> <p><a href="https://chatgpt.com/share/95fcd777-b03c-4cfe-8d1d-12e53f5a4dfe">Chat history</a>.</p> <blockquote> <p>I want to configure VM options inside IntelliJ by project</p> </blockquote> <blockquote> <p>My project uses sbt, and I want to set this in a way that I can share with my colleagues</p> </blockquote> <blockquote> <p>I am using a build.sbt file, but IntelliJ is not loading it's JVM options and I need to set this up manually. I want to automate this</p> </blockquote> <p>This is an example of my most common interaction with ChatGPT: it generally gives some options that could work, but not in my specific case. So I start to ask for more information until it eventually starts to <a href="https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence)">hallucinate</a>.</p> <p>In this particular case, the hallucination was in:</p> <blockquote> <p>The sbt-idea-plugin can generate IntelliJ IDEA project files based on your SBT configuration, including JVM options. This allows you to ensure that IntelliJ loads the correct settings from the SBT configuration.</p> <p>Steps:</p> <ol> <li>Add the <code>sbt-idea-plugin</code> to your <code>project/plugins.sbt</code> file:</li> </ol> <pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>addSbtPlugin<span style="color:#f92672">(</span><span style="color:#e6db74">&#34;com.github.mwz&#34;</span> <span style="color:#f92672">%</span> <span style="color:#e6db74">&#34;sbt-idea&#34;</span> <span style="color:#f92672">%</span> <span style="color:#e6db74">&#34;2.4.0&#34;</span><span style="color:#f92672">)</span> </span></span></code></pre><ol start="2"> <li>Run the following command in your terminal to regenerate IntelliJ IDEA project files:</li> </ol> <pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>sbt gen-idea </span></span></code></pre></blockquote> <p>Yes, there is a <a href="https://github.com/JetBrains/sbt-idea-plugin"><code>sbt-idea-plugin</code></a>, but no, it doesn't generate project files based in my SBT configuration. It is not even from the repository that it is supposed to be pointing too.</p> <p>I will just not say that this interaction was a waste of time because some of the other ideas can be applied, but this was the kind of case where I ended up just being disappointed because the thing that seemed to be a possible solution for my problem ended up not existing at all.</p> <h2>The ugly</h2> <p>There are lots of examples where ChatGPT just plain disappointed me. Here is a few:</p> <blockquote> <p>What is the language that is a fork of Go that has extra features like map?</p> </blockquote> <p><a href="https://chatgpt.com/share/2b854bbd-d376-4049-97fe-b47321c1ef07">Chat history</a>.</p> <p>This is one of those cases where I saw something once and was trying to remember. ChatGPT just gave me lots of answers that were not a fork of Go (<a href="https://vlang.io/">V</a>, <a href="https://github.com/gluon-lang/gluon">Gluon</a>, <a href="https://www.ponylang.io">Pony</a>), hallucinated a programming language (Gomini), and said that <a href="https://ziglang.org/">Zig</a> is focused in STEM.</p> <p>By the way, I eventually remember what the language was: <a href="https://goplus.org/">Go+</a>. To be fair with ChatGPT, I misremembered that the Go+ had map (as the <a href="https://en.wikipedia.org/wiki/Map_(higher-order_function)">high-order function</a>, not a hashmap that both Go and Go+ have). But still, Go+ explicit announces that it is a language for STEM everywhere in their website, and even them ChatGPT couldn't answer.</p> <blockquote> <p>In IntelliJ, I need to pass some VM options to the run configuration for my application to run tests, e.g.:</p> <pre><code>--add-exports=java.base/sun.security.x509=ALL-UNNAMED --add-opens=java.base/sun.security.ssl=ALL-UNNAMED -Dconfig.resource=api.test.conf </code></pre> <p>However, the <code>api.test.conf</code> may be <code>jobs.api.conf</code> or something else depending on the subproject I am running inside Idea. Can I parametrize this in some way?</p> </blockquote> <p><a href="https://chatgpt.com/share/aa9bd051-65de-49d9-8d56-0e28de94722c">Chat history</a></p> <p>So quick answer: you can't. IntelliJ macros are not powerful enough for this. ChatGPT itself said so when I asked about the IntelliJ macros <a href="https://chatgpt.com/share/7a057bf5-bcec-458b-9432-98e3f551ad28">directly</a>.</p> <p>But in this particular case, ChatGPT happily said &quot;yes, you can parameterize VM options in IntelliJ IDEA to make them more flexible depending on the subproject or other context&quot;. Thanks for wasting my time, I guess.</p> <h2>Conclusion</h2> <p>I do think that LLMs, in particular ChatGPT since this is the one I have more experience with, are improving. But still, if there is anything that I concluded after putting all those experiences in a post is that you still need someone that knows at least as much about whatever you're asking ChatGPT to actually validate the answer.</p> <p>Not only because LLMs hallucinates a lot, but also because even in the &quot;good&quot; example the LLM needed extra input to figure out about the possible drawback of the solution. The only case where I felt that ChatGPT outsmarted me was the &quot;amazing&quot; case, but this was literally the only example that I could find in all my chat history with ChatGPT that was like this.</p> <p>So yes, at least for now I still don't think AI will be our overlords. Maybe someday.</p> https://github.com/thiagokokada/blog/blob/main/posts/2024-09-02/01-my-coding-experience-with-llm.md Mon, 02 Sep 2024 00:00:00 +0000