18 Sep 2024 Enhancing ChatGPT Behaviour with Prompt Engineering, Function Calling & Fine-Tuning

Successfully guiding new, dynamic models such as ChatGPT towards the desired behaviour requires the implementation of specific strategies. As AI applications continue to grow, guaranteeing the accuracy of these models becomes absolutely necessary. In this article, we will explore three fundamental approaches: prompt engineering, function calling, and fine-tuning. These methods offer distinct pathways towards refining ChatGPT’s behaviour across GPT 3.5, GPT 4, and GPT 4o. Join us today to get a clear understanding of these approaches and their significance in shaping ChatGPT’s functionalities.

Text Generation & Prompt Engineering

OpenAI has trained text generation models to learn from natural language, images, and code. These models are guided through ‘prompts,’ which are text-based inputs that help shape their responses. When you ‘program’ a large language model, you’re essentially designing a prompt by giving explicit instructions or providing examples of how to achieve the desired outcome.

Prompt engineering involves rephrasing and structuring a prompt to make it clearer or more specific. This process requires the careful selection of words and phrases, as well as the application of strategies to ensure unambiguous instructions. The goal is to effectively communicate our intent, reducing the chances of misinterpretation and improving the quality of generated output. What’s more, you can manipulate some parameters to further refine the results and to achieve greater accuracy.

Parameters

You can adjust the results by changing the values of some LLM properties:

• Temperature: Controls the sampling temperature, ranging from 0 to 2. Higher values, such as 0.8, produce more random outputs, while lower values, like 0.2, result in more focused and deterministic outputs.
• Top_p: Also known as nucleus sampling, this parameter limits the model to consider only tokens within the top p probability mass. For instance, setting it to 0.1 means only the top 10% of probable tokens are considered.
• Frecuency_penalty: A value between -2.0 and 2.0. Positive values penalise tokens that have already appeared frequently in the generated text, reducing the likelihood of repetition.
• Presence_penalty: A value between -2.0 and 2.0. Positive values penalise tokens that have already been used, encouraging the model to explore new topics.

Combinations

Adjusting the temperature and top_p parameters will make the response more tailored to your requirements:

For example, if you need a structured response in JSON, there are two different approaches:

1. Adjust parameters like temperature and top_p and indicate the desired response format in the prompt.
2. Use function calling to provide a structured response.

Note that neither of these approaches can guarantee a valid response.

Function Calling

Custom functions can be created and supplied to a generative AI model through function calling. Rather than directly executing these functions, the model produces structured data output that identifies the function name and recommends the arguments to use.

Implementation

To implement function calling, you need a message and a set of functions (which could be a single function). They must follow a JSON schema as shown below:

1. Call the model with the user query and the set of functions:

let messages = [
      {
        role: 'system',
        content: 'Perform function requests for the user',
      },
      {
        role: 'user',
        content: userMessage,
      },
    ];

let functions = [
      {
        name: 'request',
        description:
          ‘Function description',
        parameters: {
          type: 'object',
          properties: {
            keyword: {
              type: 'string',
              description: "Property description",
            },
          },
          require: ['keyword'],
        },
      },
];

2. The model can choose to call one or more functions. In this case, the content will be a stringified JSON object adhering to your custom schema (and remember that the model may occasionally hallucinate parameters).

3. The functions and the message must be included in the body of the request:

const firstResponse = await makeRequestToOpenAI(res, messages, functions);

4. After receiving the response, parse the string into JSON in your code. If valid arguments are provided, you can call your function using those arguments. Here’s an example:

if (firstResponse.function_call.name == ‘request’) {
    let argumentObj = JSON.parse(firstMeeraResponse.function_call.arguments);
    content = helloWorld(argumentObj.appendString);
    messages.push(firstMeeraResponse);
    messages.push({
      role: 'function',
      name: 'request',
      content,
    });
  }

5. Once the function response is appended as a new message, you can call the model again to summarise the results and return them to the user:

const secondResponse = await makeRequestToOpenAI(res,secondMessages);
    // Final response
    const successResponse = {
      success: true,
      message: secondResponse.choices[0].message.content,
    };

Fine-Tuning

Fine-tuning involves training an existing machine learning model on a smaller, specialised dataset, with the aim of preserving the original capabilities of the pretrained model while customising it to better perform specific tasks or applications. This technique is particularly beneficial when you have a sufficiently large dataset, typically comprising 10,000 or more data points.

Fine-tuning allows you to maximise the potential of the models accessible through APIs by offering:

1. Improved Model Handling: Fine-tuning allows the model to follow instructions more accurately, such as delivering more concise answers or adapting its output to specific linguistic or stylistic requirements.
2. Increased Confidence in Output Formatting: With fine-tuning, the model can format responses more consistently, which is crucial in applications where specific formats like JSON are necessary.
3. Personalisation of Output Tone: This is an excellent way to adapt the model to reflect the specific voice, tone, and style of a brand or company, providing a more personalised and cohesive user experience.
4. Token Savings: By using shorter, more effective prompts, you save tokens, which means saving money.

Implementation

The steps to fine-tune a model are outlined below:

Prepare the Dataset

To train the model effectively, the first step is to organise the dataset. It’s essential to create a diverse range of example conversations that resemble those the model will encounter in production when making predictions.

Each row in the dataset should represent an example conversation, formatted according to the ChatGPT structure. A minimum of 10 examples is required for training, but to achieve optimal results, it’s advisable to provide a dataset with at least 50 to 100 examples.

Start with 50 high-quality demonstrations to assess whether fine-tuning enhances the model’s performance. Even if the model is not production-ready, noticeable enhancements indicate that providing more data could lead to further gains, whereas a lack of progress may indicate the need to reconsider the task setup or to restructure the data before expanding beyond this small example set.

{"messages": [{"role": "system", "content": }, {"role":    "user", "content":}]}

Token Limit

Token limits vary depending on the chosen model. For instance, with gpt-3.5-turbo-1106, the maximum context length is 16,385 tokens per example. However, with gpt-3.5-turbo-0613, each training example is capped at 4,096 tokens.

Check Data Formatting

Before using the dataset, it’s imperative to ensure that the format is correct. OpenAI provides a Python script to check for errors, token limits, and to estimate the cost of fine-tuning.

You can find more information here.

Upload Data for Training and Validation

Two data sets are required: one for training and one for validation, and both must be JSONL files. Training the model will take some time, depending on the amount of data provided, the number of epochs, the base model, and other task-specific parameters. Additionally, as training jobs are processed in a queue, the server may experience delays due to concurrent tasks from other users.

Deploy the Model

Now the model can be deployed. If the model has been customised multiple times, you can switch between different versions.

Use the Fine-Tuned Model

Once all these steps have been carried out, the model will be ready for use.

Conclusion

Prompt engineering, parameters adjustment, function calling, and fine-tuning are vital methods to refine ChatGPT‘s behaviour. Each approach offers its own advantages, allowing users to customise ChatGPT’s outputs to their specific needs. By understanding and harnessing these methods effectively, you can fully leverage its capabilities to produce accurate and contextually appropriate responses. As we progress in natural language processing, these methods will continue to be highly significant in shaping the future of AI communication and interaction.

Here at Clearpeaks we use these AI capabilities in Observation Deck, our own framework that provides top decision-makers valuable insights tailored to their exact needs and makes for an ideal unified solution offering a single point of access to all your BI content, powerful custom visualisation options, and embedded AI-powered chatbots to streamline data retrieval.

If you’re interested in learning more about fine-tuning AI models or how Observation Deck can transform your data and analytics strategy, our team is ready to assist. Contact us today and we’ll take your AI and BI solutions to the next level!