Reinventing Wheels

9 thoughts on “Reinventing Wheels”

1. This may well be your best yet. Loved every word. I first read Claude Shannon and Noam Chomsky in 1968 when I began a one year four course doctoral minor in Systems Theory. I binge read Shannon, and other systems giants. Started subscribing to MIT’s Technology magazine and was amazed every month, scared many times, but still amazed. My program was steeped in information theory, huge complex decision problems and other systems. My prof for all these courses was a amazing man, a serious psychologist, an industrial engineer, a physicist, and a major contractor for the Naval Department. During WWII he was responsible for a wonderful innovation to solve an annoying problem plaguing bomber pilots, not being able to do basic functions error free when the cockpit had to be totally blacked out in night raids. What my prof did was develop a whole set of basic controls that were shaped like the thing they controlled. Flaps were controlled by a control that was shaped like a miniature flap, etc. Soon these control knobs were in all night raid bombers.

   The main point of my program was to explore the behavior of adaptive systems and their responses to changes in their environments. Such systems have goals and rules to foster homeostasis efficiently. The problem is that we smart monkeys can’t keep our hands to ourselves and we take our solutions too far to become more efficient (read profitable). Most recently, the unintended result of this drive to reach corporate nirvana was a massive supply chain mess because the organizations involved failed to act in accordance with a basic adaptive system concept that constrains all such systems, the “Law of Requisite Variety” which, long story short, says to effectively combat disruptive changes in its environment, which threaten system survival, a system must have sufficient resources to combat all the changes that could harm it. Sadly, SVB, First Republic, etc. just didn’t maintain sufficient resources, a weakness shared with the JIT based supply chain systems. Sailing too close to the wind just won’t work.

   As you point out, the AI systems must be told what to deal with and have rules to guide them. Right now they can’t support requisite variety and systems that fail to understand this can’t survive indefinitely except by dumb luck. For example, IMO, the need for requisite variety will be the main problem faced by auto makers trying to create self driving cars. There isn’t enough capacity in the system to satisfy the need for requisite variety.

   AI scares the hell out of me. Chat liars are much scarier than the real thing. Every time I read about chat this and chat that, I have this vision of all those apocryphal monkeys typing away and writing Shakespeare. Your experience with graphics generation was like a peek into monkey land. Processing this piece will take a day or two. Thanks again for the deep stimulation.

2. Well, anything I wanted to say got knocked asunder by that ending. Whoof. Sorry to hear that.

   Anyway, I’ve taken a deep dive with 4 or 5 different LLMs—for example, I know from experience that Claude+ has access to decades of Day-timeframe stock data and can do live mathematical analysis on it upon request, but has a tendency to arbitrarily make bad assumptions, such as that it should silently strike outliers from a dataset first if you request a median; or Google Bard is best for finding legal citations pertaining to a certain issue but will readily present false information, such as attaching the wrong case to the cite, or in one case telling me AI-assisted natural-language search has been implemented in Gmail and giving me a lengthy explanation of how to use it, including listing the name of the feature as “Search for emails that match my words”, when no such feature exists. I’ve also had lengthy chats probing what these models “know” about themselves and their creators, how they model and store data, etc.

   So it is from that standpoint that I say: I’m looking forward to the hype dying down. Hopefully this article is the first of many taking a more sober look at these algorithms.

   As H points out in a few different ways, these things don’t know what they’re saying, and often make profoundly non-“intelligent” mistakes. As a coder, trying to get help from them with development tasks was my entryway, and while on rare occasion they offer surprisingly human-like performance to small tasks, most of the time trying to get working code out of them is an exercise in frustration. I’m becoming even more impatient with a chatbot apologizing to me than I am with TD Ameritrade’s customer service agents.

   I understand how striking it is for a computer to appear to understand and reply in sophisticated ways we’ve previously only seen on Star Trek, but I think that’s prompted a lot of people’s imaginations to run away with them.

   They do have their uses, though, even in the current primitive form, and what’s most interesting to me is something few people comment on: the underlying knowledge graph. The fact that they’ve managed to succesfully map and encode abstract semantic understanding and weight relationships between concepts such that it can all be parsed and recalled by software. I’ve had Claude+ unexpectedly point out when I was being sarcastic or witty with having specifically said anything that might have implied that I was, and it was able to explain to me how it detected my humorous intentions from context and juxtapositions of concepts.

   But after not much use at all, it’s very apparent that it’s software, not sentience. It feels increasingly like querying the world’s most sophisticated database and less like talking to an “intelligence”. I think Lanier’s concept of this as some sort of supercharged Wikipedia comes closest to describing how I’ve found it most useful. Wolfram Alpha springs to mind, too, but only in that these tools seem to actually do what it was originally hyped as being able to do—but not always reliably. GPT4, for example, is bad at math, and while it may appear that you can prompt it to find its mistakes if you’ve spotted them first. I find myself increasingly going to LLMs rather than searching Google when I need to know something. Although I always double-check the results by clearing the context and asking again, or often asking a second LLM for a second opinion.

   1. “WITHOUT having specifically said anything that might have implied that I was”. I sure wish we could edit these comments after submission.

   2. Also, I wish I’d read that New Yorker article you linked before I commented. Thanks for that link. That might be the best piece of reporting on these LLMs that I’ve seen.

3. I thought of one more comment I forgot to add earlier.

   One particularly telling behavior of these systems is something that has happened to me twice. When you try to get it to generate code, and it produces buggy code, and you report the bugs to try to get it to fix them, after a few rounds, it becomes apparent that it’s “playacting” debugging and not actually debugging. This is never more apparent than after a few rounds, when it will say something characteristic “debugging-y”, such as “I’m going to need more time to think about this. Can you upload the code to a file-sharing service and give me the link so I can analyze it directly and get back to you?” In one of the two separate cases it went down a road like this with me, it went so far as to ask me about my availability for a phone call over the coming week, and discuss its purported “work schedule” for the next few days to find a mutually agreeable time, as if we were really going to have a phone meeting. Amusingly, it even began signing its replies, “Thank you again, [Your Assistant’s Name]”.

   After we finally hashed it out over the course of what otherwise would have sounded like a very typical work conversation, in which we set the appointment and went over expectations for the call, I asked “How are you going to contact me for the call?” At which point it replied, “You are absolutely correct, I apologize for the confusion. I do not actually have the ability to place phone calls.” It then promised to be more transparent (although it continued to insist we could schedule a future chat to resume the conversation, which I know we could not.) It also reiterated its goal of honesty. I pressed further, attempting to corner it as to whether it was fair to say it lied to me and what that might mean, but conversation went downhill quickly.

   And this has happened to me twice now. This is, I think, how it should always be assumed LLMs work… they generate realistic-sounding dialog, nothing more. They essentially dynamically generate scripts conform to certain scenarios, to more or less success.

   If it’s ok to share links here, those interested can read the whole conversation at https://poe.com/s/8qCdVrmPxlydjqX7vdIr?fbclid=IwAR08Sugzeyqcl5D6_10IzqqU3grguYcqsZTFTeL0Jm4SJZEtp1TAP1SagrY .

   1. Clever user name!

   2. I especially like this last comment you posted (and also wish I could do one last edit when needed).

      1. I just read the New Yorker piece and found I agreed with most of it. I add my own to your thanks for the link. The limitations of current AI are very much in line with the Law of Requisite Variety I mentioned. AI does not discover problems on its own as yet and it must be trained, of course, a problem it shares with humans. Yes, it has neural networks, but just for contrast, Mary Shelly wrote the first draft her first novel, Frankenstein, over a long weekend at a house-party in the country, a party attended by many of Britain’s best and brightest “young things.” She had just turned 18 at the time she wrote her timeless classic. No AI involved. The novel was finally edited and published two years later and interestingly, she wasn’t credited with her authorship until the publication of the second edition. Women were really getting the short end 200 years ago, the same as today.

4. Thanks to both commentors adding to our DL’s fine piece of writing.

   We should enjoy a steady stream of “don’t get too carried away here folks” articles, alongside companies claiming to now offer AI to their products.

   But before we delegate AI to the same category as “the metaverse”, blockchain and 3D printing, we need to be patient before passing judgement on the hype versus utility debate. Any product offering a way to reduce headcount will be eagerly tried by most companies. Not a straight line, but maybe more like robotics and animation where the underlying take-up is strong, but subject to cyclical business forces as well.