There has been a lot of talk of government regulation of AI and a lot of fear mongering that AI is going to take away 95% of jobs. These fears are overblown. Yet there still are worries to AI. Before I get into the possibilities as well as the dangers. It’s worth a brief overview of how AI like chat gpt actually works, which is being lost in the current dialogue around AI in general.

The Economist published a great piece discussing the intricacies without overwhelming laypeople with details in a recent edition of the magazine. I’m going to summarize the basics below, which help to point out some of the holes in the thinking around fears of AI and the threat it poses.

How ChatGPT Works

First of all, credit is due to the creators of this technology, which is in the process of synthesizing almost all human knowledge. Just the internet itself was a great leap forward in access to knowledge. As Elon Musk remarked on a recent episode of Real Time with Bill Mahr, someone accessing the internet via Starlink from the Amazon Rainforest today now has access to more information than the President of the United States did in 1980. ChatGPT itself embodies more knowledge than any human has ever known.

GPT-4, the latest iteration of ChatGPT is a Language Learning Model or LLM. LLM’s are derived from “Deep Learning” models first developed in the 2010’s. This used a mix of huge datasets and powerful computers running graphic processing units or GPU’s to improve computer’s ability to recognize images, process audio and play games.

Due to the model’s ability to associate and group words, concepts and other details of media, it is also referred to as neural networks. These neural networks though, have long been embedded in software that had other functionality, think of Gmail using this to predict what words or sentences you will type next. ChatGPT now let’s humans interact directly with this technology rather then experience it as an ancillary functionality of another tool.

When entered into ChatGPT, the words typed are converted into representative sets of numbers which are assigned to a token. Tokens can commonly occur together and can be things like words, affixes (like dis or ised) or punctuation. GPT-3’s dictionary contains 50,257 tokens.

These tokens are then grouped together into a “meaning space” of words with similar meanings that tend to be grouped together.

The result after retention of the data is a series of probabilities, for each token in the LLM’s vocabulary, of that word going next. The model chooses one. Then repeats the process.

The model weights each pair of tokens, showing how much to “pay attention” to one when processing the other. This bakes in an ability to form connections.

GPT-3 is able to process a maximum of 2,048 tokens at a time, about the length of a decent newspaper article. GPT-4 can handle inputs of 32,000 tokens long. The more text it can take in the more context it can see but the computation rises exponentially with the length of the input

Without data, the LLM is not very useful. Before the model learns from test data, asking it questions may spit out random words with little connection to each other. So it needs data to learn from. It needs to go through data to start to associate word groupings with each other. It’s easier to think of the LLM more of a statistical representative sample of the web as opposed to an all knowing being thinking on its own.

As The Economist noted:

GPT-3 was trained on several sources of data, but the bulk of it comes from snapshots of the entire internet between 2016 and 2019 taken from a database called Common Crawl. There’s a lot of junk text on the internet, so the initial 45 terabytes were filtered using a different machine-learning model to select just the high-quality text: 570 gigabytes of it, a dataset that could fit on a modern laptop. In addition, GPT-4 was trained on an unknown quantity of images, probably several terabytes.

The Issues With LLM’s

The LLM model can be adjusted to not simply spit out the most statistically probable next word, it can be varied by human commands to create a form of “creativity” but it’s important to understand what this is and is not.

What many may be missing is that these LLM’s will end up producing results that mirror what is on the internet over a short period of time, ie the test period. This will contain all the issues of the internet which already existed within that data.

For example, just because something is more likely to be printed or exist in the internet does not mean it is a representative sample of what exists in the real world. According to Statista, 58.8% of the internet sites on the net were in English while English speakers represent 4-5% of the global population. This brings with it cultural, linguistic, national and political biases with it, among other biases it would have within that data itself as compared to other languages.

Without guiding the LLM with some sort of weighting, be it what is more statistically significant, or some variation of that for “creativity” (what does that mean really? Picking the 4th most popular subsequent word every 7 iterations?) the model would struggle to choose what to repeat.

This can be most easily seen in image generating AI. Ask AI to create whimsical fictional scenes and it has little issue. Likewise, ask it to create images based off of famous figures that have been photographed immensely then it has little problem mashing up images.

Prompt the AI to generate an image that requires open interpretation though and it fails miserably. Think of a relatively open ended prompt such as “a thousand of couples celebrate in Iceland in summer, dj ,deep house huge theater dancers” and the image generators seem to produce hybrid pictures of what it thinks a human should look like which may look like grotesque creatures out of a horror movie (see below for this same prompt).

Dig a little deeper and you can see how open ended prompts like this can cause the AI problems and put creators in an uncomfortable space. Do the training images most depict white people? Then left to it’s devices, the AI would produce mostly white images. How should the people above be dressed? What proportion should be men or women? None of this is clear and images produced are only likely to reflect the training images the model was trained on.

Many of us were surprised or excited to see AI produce images of things that never happened such as Trump being arrested in the streets of New York or the Pope with some urban outerwear but this is relatively easy for the AI because their faces were likely well documented and in the training data. A prompt of Morgan Freeman by a user the other day produced something similar and lifelike.

Uploading your own picture to Midjourney, which converts text to images seems to solve some of these problems but it does point to a bias the software already has in the data towards recognizable figures.

The creators of ChatGPT themselves acknowledge the bias problem in their own research via huggingface.co:

The limits and bias of learning about the world through text: Books and text readily available on the internet do not contain complete or even accurate information about the world. Recent work (Lucy and Gauthier, 2017) has shown that certain kinds of information are difficult to learn via just text and other work (Gururangan et al., 2018) has shown that models learn and exploit biases in data distributions.

What this tells us is that ChatGPT and other LLM’s are more like mirrors of the internet at a snapshot in time and all the problems as well as great free information that comes with that. Users should keep pitfalls like this in mind when interacting with the software.

Other worries are more high level and philosophical. One fear is that AI could act as some idiot savant that destroys humanity. This is embodied in the paper clip maximizer thought experiment:

If you told a machine to maximize the number of paperclips it produces the machine would eventually start destroying things like computers, refrigerators, or really anything made of metal to make more paper clips once other sources of metal run out. This concept has been coined as instrumental convergence.

While this is a worry for the future, the bias of the training data is staring us in the face now, and users would be wise to consider this when utilizing LLM output.

The Advantages of AI

Despite the worries, the advantages of AI are compelling. I have spent the past few weeks cataloging what I can of the numerous websites and apps popping up which use AI to enable users to create much more than novices have had the chance to without technical programming skills.

These in life but are not limited to:

• Making copyright free music instantly.
• Copyright free images instantly which are contextualizad to any story.
• Video editing – I saw one which would edit and auto cut the shots for 3 different camera angles to who is speaking and changing perspectives. This work previously took an editor hours.
• Producing content – Sites like Ryter.me can produce articles with content that can be edited and individualized by authors.
• Charts – Chartify.ai can make charts from jumbles of data like a CSV file and produce various graphs of the data in seconds. The author just chooses which they would like.

These are just a few of the many examples. From the point of view of writer, rather than fear AI will take my job (I don’t get paid for this anyway) AI could actually multiply my productivity in an astounding way.

Various software could help me create draft posts and edit them to my liking, create original artwork which speaks to the content of my writing, add audio so people can listen to the posts and build a new site to further organize and categorize met past work.

In fact, I have already started to do this with using AI to create the image graphic for this post (I asked Midjourney to create “AI destroying the universe”).

Rather than worrying that journalists or writer will get laid off, we should be excited of the further empowerment of small creators. Now a small team of motivated people can likely produce their own news site or provide daily updates to their blog rather than weekly. Authors could pump out books on a monthly basis rather than once or twice a year.

Only a year ago I was gathering draft writing for a book and I spent years writing a few hundred words every day. Now I can use the tools of AI to put out a book in a few weeks. Rather than a threat, seeing these as tools will allow some people to vastly increase their creative and academic production and that is a huge boon for humanity and human knowledge.

Rather than striking fear, these tools should be seen as a great opportunity to produce more than ever. However imperfect they are, for the moment they still have enough promise to outweigh the inherent bias and future issues they may cause.

The information provided by www.cashchronicles.com is for informational purposes only. It should not be considered legal or financial advice. You should consult with an attorney or other professional to determine what may be best for your individual needs. www.cashchronicles.com does not make any guarantee or other promise as to any results that may be obtained from using our content. No one should make any tax or investment decision without first consulting his or her own financial advisor or accountant and conducting his or her own research and due diligence. To the maximum extent permitted by law, www.cashchronicles.com disclaims any and all liability in the event any information, commentary, analysis, opinions, advice and/or recommendations prove to be inaccurate, incomplete or unreliable, or result in any investment or other losses. Content contained on or made available through the website is not intended to and does not constitute legal advice or investment advice and no attorney-client relationship is formed. Your use of the information on the website or materials linked from the Web is at your own risk.