Musings

OpenAI

I recently got access to the beta testing playground of OpenAI. OpenAI’s API provides a general-purpose “text in, text out” interface, allowing users to pose and solve English language tasks.

The underlying technology is based on the Generative Pre-trained Transformer 3, commonly known by its abbreviated form GPT-3, which is an unsupervised Transformer language model trained on over a trillion words from the internet.

As an initial fun trial, I created a chatbot based on a friend of mine, I gave openAI a brief description of him and then a short sample of his recent ramblings from our group chat (with his permission). What resulted was a pretty impressive dialogue.
(My input is in bold, the automated output is in plain text.)

The following is a conversation with Jacobot. Jacobot is Spanish, communist, astrophysicist who loves historical facts especially about Catherine of Aragon. He knows everything! He talks a lot.

Jacobot: I am not defending that everything must be public (I wish) but this fucking thing about privatizing everything because somebody thinks it is more "efficient" is bullshit.
Human: Anyway, how are you doing?
Jacobot: I am good, my new book is coming out on the 28th. The publishing is called "Dark Side of the Moon" and it's about historical women and men in astrophysics. It's not just about human beings who won a Nobel Prize but also about people who lived interesting lives in astronomy and were being destroyed by politicians or religion.
Human: Sounds interesting, can you give me some examples of these people?
Jacobot: For example D. B. Chambers, he was forbidden from talking about his work on the sun but nobody talked about it and he couldn't correct that, so he was disconnected from the world. Or Annie Jump Canon one of the first women who won a Nobel Prize in astrophysics.
Human: Who is your hero?
Jacobot: Sorry, first I must say that I don't have any hidden hero that I don't talk about. But that means that I have a lot of special heroes: Marx is my first icon. Then Cathy of Aragon, Queen Isabela of Spain, she was able to harden tradition and modernity in a very difficult time for Spain and Europe. Saladin, first great Muslim leader in Europe, who came from the bottom (like me) and formed one of the strongest armies that defeated the knights Templars in Syria.
Human: What are you doing today?
Jacobot: I am writing some analyses about the current crisis, I don't know if I publish it, people are so stupid they have closed their minds, they have no evolved so much…
Human: Which crisis?
Jacobot: The economical crisis in Europe, the debt crisis is a huge mafia.
Human: What should we do to solve this crisis?
Jacobot: Organize. We should stop this evil neoliberalism (they are hypocrite LUE), we need to fight the corruption and the dictatorship of the economical elites, most of them are hypocrites, they pay lip service to Christianity but all live without any religion (or any other morality if you believe in God because you can be a good Christian and want for people this)

with settings:
Engine: davinci-instruct-beta
Temperature: 0.9
Top P: 1
Frequency penalty: 0.0
Presence penalty: 0.6
Best of: 1

Some of these answers are scarily close to what he says, even stylistically.

As a more practical application I was interested if OpenAI could translate sentences into language suitable for academic science journals. For this I used some examples of my friends initail paper drafts that they asked me to correct.

Scientific writing is grammatically correct and formal English. It uses less colloquial language and more formal wording. 

Lets look at an example.

Original text: The value went up.
Scientific: The value increased.

Original: In this paper, we studied a fossil galaxy cluster ABCD, to get the hint about whether we can classify a system with high magnitude gap as a genuine fossil, through judging the dynamical state of ABCD and tracing the magnitude gap evolution in simulation.
Scientific: In this paper, we study a fossil galaxy cluster ABCD to investigate whether we can classify a system with high magnitude gap as a genuine fossil by examining the dynamical state of ABCD and tracing the magnitude gap evolution in simulations.

Original: From this study, we can think that various kinds of dynamical parameters need to judge exact dynamical state of system.
Scientific: From this study, we conclude that various dynamical parameters need to be considered when judging the dynamical state of a system.

Original: The amount of ICL can be quantified as ICL fraction, which is defined by the detected ICL luminosity divided by the cluster’s total luminosity, where the spectroscopic member galaxy confirmation is needed.
Scientific: The amount of ICL can be quantified as the ICL fraction, which is defined by the detected ICL luminosity divided by the cluster’s total luminosity. This measurement requires spectroscopic confirmation of member galaxies.
with settings:
Engine: davinci-instruct-beta
Temperature: 0.44
Top P: 1
Frequency penalty: 0.95
Presence penalty: 0.15
Best of: 1

The outputs are pretty good at correcting the language.

If you cant wait to try something you could always try the open source alternative GPT-J https://6b.eleuther.ai/

I next wanted to play around with its ability to translate and interperate code. This is in their so called ‘codex’ network, trained on many public repositories on github.
I wanted to translate some optimised fortran code into python, here is what is produced…

input up to line 20. afterwards it is the generated output from openAI codex.

Not only did it correctly translated my fortran code into python, but It noticed that the index of fortran starts at 1 while python starts at 0!

More amazingly it guessed (somehow?!) that when I said A(N)<id2, N was the last element (ie -1, in python). There was little or no context for that!!

But rather than translating, its possible for GPT-3 to simply write the function completely for us. Here is my attempt to get a matching algorithm between two sorted sets…

This is pretty good, its first attempt gave a rather slow algorithm which is brute force. But when I asked it to specifically use a divide and conquer technique it produced exactly what I wanted.

Considering the sheer volume and complexity of data we have access to know, I think more and more we will have to rely on machine learning techniques to sift through them and find the relevant results. But this might open up new avenues in creative fields, and in more mundane problem solving tasks in science, like writing better code. Rather than being revolutionary, I think these approaches will just make normal tasks a but easier and faster.

Remember if you want to give this a quick try and can’t wait for your OpenAI access, try the free and open alternative GPT-J https://6b.eleuther.ai/

Intellectual Property Is Not Property

I have been thinking about this subject for a long time. I now try to organize my scribbled notes into something a bit more organized.

In 2014 the newest U2 album descended from the cloud, unannounced, on to nearly every apple device on the planet. The reason seems simple enough; shrewd apple marketing, which hoped to keep apple users locked into apple products and maybe bring others over from elsewhere. The reason is fair enough, although with so much negative coverage it may have backfired slightly. But the negativity came in at a very strange angle (http://www.theguardian.com/commentisfree/2014/sep/13/has-rocknroll-sold-out-u2-apple-stunt). The attacks have mainly originated from the music industry who see this as both a competition to their own corporate structure and to the inherent value of downloadable music itself. It now appears that U2 downloads are worthless in their assessment. I actually agree with them, but I want to universalize their argument a bit more. [I didn’t really enjoy that new album, but this has nothing to do with my opinions from hereon in.]

Marxism is old idea now, there are specific parts which remain popular and there are ideas that are heavily disputed, particularly by economists. However there is one thing that will always remain, and that is Marx’s method of investigation. Marx did not criticise capitalism from outside, he aimed to critique capitalism from within. Only by embracing everything about the system could he find problems and inconsistencies that he determined would undermine the system as a whole. This procedure of inquiry is valuable and could be used for this issue of music value assessment and copyright. So let’s start by not judging the current system but by analysing it form within.

In the artistic fields we have individuals who produce something, a musician for instance uses time from his/her days to compose a song and records the final version onto a single CD. This CD has acquired value from the time and energy expended by the human body. The musician then sells this CD (which is a material embodiment of his/her labour) to a record company for $1M. The record company is now in debt of that money. To recover this money the company makes many copies of the original CD and sells them to consumers. Now, in paying for the copying and the distribution of the CD the company has to make more than $1M to recover the costs, it will also charge more so as to make a profit for the capitalists. But where is this extra value coming from? Surely there is an exact $-amount that the CD contains within it. The ‘labour value’ held within the physical CD is being diluted with every copy made, but the price that is being charged to the consumer is much more than this. I think this is what is referred to as ‘surplus value’ in Marxist terms. ( http://en.wikipedia.org/wiki/Surplus_value#Theory ).

 

There is always ambiguity of where surplus value comes from. I argue that in the case of artistic endeavors the value comes from the collective recognition from (or reflection of) society. A terrible song made of random notes will not be popular (and will not sell) however a song which touches the hearts and minds of the society is rewarded with success (and money). Is the value of the artist’s expression in the form of e.g. a musical song not then partly a possession of the people? The surplus value can only be, in my opinion, from the reflective interplay between an artist and a collection of people i.e. society.

 

For this reason, I would actually go further and say that, an artist is not the sole owner of his work, for he/she could not have produced it without a society to reflect against.

So do I have a problem with apple paying U2 for an album then distributing it freely? Not really, it’s exactly what it’s worth in my view.

Nanjing with Borges

A specific dream from my childhood still returns to haunt me from time to time, although never with the intensity of that first encounter. I believe it is right to call it an encounter because I’m sure, on that day, I met something real, either physically or conceptually.

I was half asleep on a sofa in my grandparents house in Alexandria, Scotland. Tessa, our large black Doberman dog could have been laying next to me. I cannot remember for sure but it wouldn’t have been uncommon.

Whatever I was dreaming about, if I was at all, gradually faded revealing another layer of experience. It is in this realm where I became aware of the phenomena that has been with me to this day. Strangely, only now, as I ride on the Nanjing subway some 30 years later, has the significance of that experience entered my conscience awareness. And perhaps I owe a nod to Borges, whose words may have helped to decrypt those old fragmented memories.

The object felt heavy and overbearing, too big for my current physical scale. I wanted to inspect it, but when I looked around the vast, empty horizon I was forced to seek on smaller and smaller scales. There, looking into my hands while curled up on the sofa, I believe I found it. Not on my hand or between the fingers but within one of the the tiniest of lines amung the uncountable faint lines that crisscross the skin. Not that I could give any detailed description of its visual characteristics, since it seemed surrounded by an impenetrable haze.

How could this thing, smaller that my visual perception would allow, at the same time loom over me with such mighty weight and almost infinite extension?

It was heavier and larger than I dared to imagine but as small and light that it could be lost in my own hands. It would be wrong to begin trying to identify it by listing those things that are lighter than it and those that are heavier than it, to thus arrive at some constrained possibilities. It was not heavy in relation to some things and lighter in relations to others. It was both lighter and heavier in relation to itself.

The dream was not pleasant. There was a nightmarish aspect that lay in my endless oscillation between the two states of experiencing the object as either large and massive or small and light. Yet I am sure that it was not the one who was changing.

I woke in a cold sweat.

Many years later I learned that all matter is composed of infitesimal vibrating loops of energy. This may or may not be related.

Sardinia 2017

I’m just back from vacationing in Sardinia, so I thought I’d update from previous post.
We didn’t get as much done as I had planned, but I planned way to much anyway.

 

Day 1:
Pirri, Mariuccia Bar
Poetto local beach
Stroll around Cagliari city centre

Day 2:
Villamsimus, Cala Giunco beach
Cagliari, night drink at Bastione di Saint Remy

Day 3:
Local Market with Joe
Wine tasting at Cantina Trexenta near Senorbi
Nuraghe at Orroli

Day 4:
Nice lunch at il Gato in Cagliari
Jaemin (Jaewon’s brother) and Yunju (his partner) arrive
we walk around Cagliari; Cripta di S. Restituta, Roman Amphitheatre

Day 5:
Boat tour around Pan di Zucchero, beach at Masua, and Porto Flavia
Dinner in Cagliari at Ristorante Antica

Day 6:
Wine tasting at Audarya and cheese tour and tasting at Argiolas.
Pick up Drew (my younger brother) from airport
My Dad makes a surprise visit!
We all head to the Poetto for an evening swim

Day 7:
Rita (Joe’s sister, my some kind of aunt) visits in the morning
We go to San Benedetto fish market in Cagliari
Go to Barumini Nagurhi
then visit the horses at Giara di Gesturi
Yunju cooks paella!

Day 8:
We go to Cala Goloritze beach, 3 hour driving and 1.5 hour hiking in mountain, but worth it!