Create your own AI assistant with Nevermined and Llama Index

Creating an AI assistant is all the rage. But then what?

This is the second post of a series demoing how Nevermined is rocket fuel for the monetization of both AI and data.

In previous posts (“Bringing online subscriptions into Web3 with NFTs” and “NFTs are dead. Long live NFTs!”), we spoke about how blockchain can enhance the utility of digital assets. And in Monetizing AI services with Smart Subscriptions we showed how you can use Nevermined to monetize a web service using the OpenAI API.

For this post we take on the role of an AI developer, who uses Llama Index to connect data with an LLM. What we show are the benefits of also using Nevermined: the developer will use our app to register the data (for attribution) and to tokenize access to the AI Assistant for monetization.

The objective is to create an AI Copilot that summarizes Elvis songs and that can be made available via a Discord bot to subscribers only. 

It’s one for the money, two for the show, three to get ready, now go, cat, go….

---

Assuming that we have access to the Elvis lyrics, we will describe the following 5 steps:

1. Register the lyrics with Nevermined.
2. Load the lyrics in a valid document store.
3. Create a summarizer AI using Llama Index. This will provide a protected HTTP service enabling song summarization requests.
4. Register the Elvis Copilot Nevermined and token-gate it with a Smart Subscription.
5. Pretend that we’re an app developer that buys our Elvis Subscription to configure an Elvis Discord bot.

A full overview of the end result looks something like this:

Step 1: Let’s get the data in order

As in every data pipeline, the first thing we need to do is to take a look at the data we are going to use and then load it somewhere.

We want to register all the files we will provide to the summarizer service. The benefit of using Nevermined is that it allows further tracking of where the Copilot service got the data from to provide a response, cross referencing answers with the lyrics provided.

This can be done via direct registration of all the assets using the Nevermined App as we saw in the Monetizing AI services with Smart Subscriptions post.

In order to register our data (the lyrics), we first need to create a Subscription plan. This defines the access control to any assets that are attached to this Subscription. In step 4 we’ll also attach our Elvis Copilot service to this subscription, to define the conditions under which our Elvis service can be accessed.

Creating a Subscription plan means we deploy our own smart contract and register a Nevermined subscription attached to it. Let’s say the subscription is valid for 1 year (more or less 15 million blocks on Polygon) and only costs 1 USDC, because Elvis lyrics are essential information that should be easy to access for everyone. 😉 

Now that we have the subscription, let’s register some lyrics and ‘attach’ them to that Subscription.

Step 2 — Load the lyrics in the documents store

Once the lyrics are registered with Nevermined, let’s add them to the document store and connect the AI Copilot to it. In this example, we used a fork of the ChatGPT retrieval plugin on top of Redis as the document store. 

OpenAI created the retrieval plugins project as a nice way to try to “standardize” how external data can be consumed by an AI. That makes it easier for AIs, like ChatGPT, to query external data, in case they need/information outside of their source training corpus.

The retrieval plugin exposes an HTTP API that allows indexing documents by sending an HTTP POST request. It’s only necessary to send one request per song to index in our document store and we’re good to move to the next step.

Step 3 — Create a song summarizer with Llama Index

Now that we have the data in place, let’s connect to the summarizer. We are going to use Llama Index to connect an LLM with some external data. In our case it’s perfect, because we can connect Llama Index with our Elvis document store and start sending queries to it.

Llama Index provides an extensive collection of plugins that can be used to connect your document store with your LLM (in this case OpenAI ChatGPT) and create indexes that work as a context for further queries.

What we did is create some code to expose a HTTP Rest API that receives a summarization query about an Elvis song. Next, the AI service identifies the relevant song from the data store (our ChatGPT retrieval plugin) and then summarizes the content of the song.

You can find the full code, connecting our Elvis retrieval plugin here.

We’re getting somewhere! Our AI summarizer service is returning good summaries of Elvis songs. Because we included the endpoint as part of the ChatGPT plugin, this is protected and only accepts requests from authorized users providing an HTTP Authorization Bearer token.

curl -X POST https://chatgpt-plugin.mumbai.nevermined.one/ask 
-H 'Accept: application/json' 
-H 'Authorization: Bearer xxxxx' 
-H 'Content-Type: application/json' 
-d '{"queries": [{"query": "Adam And Evil","filter": {},"top_k": 1}]}'

Now, this is where Nevermined gets even more interesting. We don’t want to share our authorization token into the wild. Instead, we’d like to offer it for sale, so only subscribers can get access to our Elvis Copilot service …

Step 4 — Register the summarizer with Nevermined

Let’s recap, we have the Elvis lyrics registered into Nevermined, associated them with a subscription and indexed them in our document store. We are exposing our Elvis assistant via HTTP. The summarizer is protected and only can be queried by authorized users.

Now we just need to connect the dots.

To facilitate the usage of this Elvis assistant service to third parties, we can register it into Nevermined and associate it with the Smart Subscription we created before. This will inherit the access conditions set in the Subscription plan: 1 USDC for 1 year of access.

In our case we deployed the AI service in a cloud environment. As you can see in the above image, we provided the endpoints that are accessible to our users, and the access token with the upstream Elvis service. This access token is never shared with the end users: it’s encrypted and only accessible by the trusted proxies selected by the user. It can be revoked at any time by the AI Service Publisher.

At this point, the access to our Elvis Copilot is protected by the Smart Subscription. It can only be accessed if a User demonstrates that they hold the access token.

Step 5 — Purchase access & create an Elvis Discord bot

Let’s switch roles.

Now we’re an app developer and we’ve found the Elvis Copilot on the Nevermined marketplace. We want to use it to create an Elvis Discord bot.

After purchasing the access token, we can send a request to the Elvis service through the service proxy using the unique JWT.

$ export REQUEST_DATA='{"queries": [{"query": "Adam And Evil", "filter": {}, "top_k": 1 }]}'
$ export JWT_TOKEN='Get the token from the Dashboard!'
$ curl -k -H "Content-Type: application/json" -H "Authorization: Bearer $JWT_TOKEN" -d "$REQUEST_DATA" https://proxy.nevermined.network/ask
{"response":"nThe song is about a person who is in love with someone who is not perfect, but they cannot live without them. [ ..]","doc_id":"099632b5-d4d4–482b-baef-77248c1dfb4c","embedding":null,"doc_hash":"451d68b33de1e8034e48c6a98865364e52edd02837f06c34c662ba6d6d462c76","extra_info":null,"node_info":{"start":0,"end":1030},"relationships":{"1":"did:nv:9db3b3c02bfed7449f9824a5bffe78c348b484ed73257c4ff14d5e0307ffa2ea"}},"score":0.7585169416635178}],"extra_info":null}

As you can see, a User (with the right Subscription) can query the Elvis assistant and get answers showing the JWT.

Sending queries through the command line wouldn’t be something Elvis “approves”, so let’s do a final bit and connect our Copilot with a Discord bot.

This bot can be integrated into any Discord server and summarize Elvis songs, so via the ‘/ask-elvis’ command you can ask Elvis about any of the songs part of the Nevermined subscription.

If you want to see it in action, feel free to join the Nevermined Discord server and ask Elvis!