GenAI

The GenAI module provides a unified interface for working with LLMs while doing conversational analysis in ConvoKit. The current implementation supports multiple providers including OpenAI GPT and Google Gemini, but is designed to be extensible to LLMs from other model providers and local models. This module makes it easy to integrate AI-powered text generation into your ConvoKit workflows for diverse tasks. The module handles API key management, response formatting, and provides consistent interfaces across different LLM providers.

The module includes a ConvoKit transformer that allow you to apply LLM processing directly to corpus objects at different levels (utterances, conversations, speakers, or entire corpus), making it seamless to integrate AI analysis into your conversational data processing pipelines.

Example usage: GenAI module demo.

Overview

The GenAI module consists of several key components:

  • LLMClient: Abstract base class that defines the interface for all LLM clients

  • LLMResponse: Unified response wrapper that standardizes output from different LLM providers

  • Factory Pattern: Simple factory function to create appropriate client instances

  • Configuration Management: Centralized API key and configuration management

  • Provider Clients: Concrete implementations for different LLM providers (GPT, Gemini, Local)

  • GenAI Transformers: ConvoKit transformers that apply LLM processing to corpus objects

Basic Interface and Configuration

class convokit.genai.base.LLMClient

Abstract base class for LLM clients. Used as a template for all LLM clients.

Provides a common interface for different LLM providers (GPT, Gemini, local models, etc.). All LLM clients should inherit from this class and implement the required methods.

abstract generate(messages, **kwargs) → convokit.genai.base.LLMResponse

Generate text from the LLM.

Parameters

  • messages – Input messages/prompt for the LLM. Can be a string or list of message dicts

  • **kwargs –

    Additional parameters for generation (temperature, max_tokens, etc.)

Returns

LLMResponse object containing the generated text and metadata

stream(messages, callback, **kwargs)

Stream text generation from the LLM. Notice that this is not supported yet.

Parameters

  • messages – Input messages/prompt for the LLM

  • callback – Function to call with each generated token/chunk

  • **kwargs –

    Additional parameters for generation

Raises

NotImplementedError – If streaming is not supported by this client

class convokit.genai.base.LLMResponse(text: str, tokens: int, latency: float, raw: dict)

Encapsulates the response from an LLM client.

Contains the generated text, token usage information, latency, and raw response data.

Parameters

  • text – The generated text response from the LLM

  • tokens – Number of tokens used in the generation (may be -1 if not available from the LLM output)

  • latency – Time taken for generation in seconds

  • raw – Raw response object from the LLM

class convokit.genai.genai_config.GenAIConfigManager(path: Optional[str] = None)

Manages configuration for GenAI clients, including setting and accessing API keys.

Handles loading and saving of GenAI related configuration data, with support for environment variable overrides. Provides a centralized way to manage API keys and other configuration settings for different LLM providers.

Parameters

path – Path to the configuration file (default: ~/.convokit/config.yml)

get_api_key(provider: str) → Optional[str]

Get the API key for a specific provider.

First checks environment variables, then falls back to the configuration file.

Parameters

provider – Name of the LLM provider (e.g., “gpt”, “gemini”)

Returns

API key if found, None otherwise

get_google_cloud_location() → Optional[str]

Get the Google Cloud location.

First checks environment variables, then falls back to the configuration file.

Returns

Google Cloud location if found, None otherwise

get_google_cloud_project() → Optional[str]

Get the Google Cloud project ID.

First checks environment variables, then falls back to the configuration file.

Returns

Google Cloud project ID if found, None otherwise

set_api_key(provider: str, key: str)

Set an API key for a specific provider.

Parameters

  • provider – Name of the LLM provider (e.g., “gpt”, “gemini”)

  • key – API key for the provider

set_google_cloud_config(project: str, location: str)

Set Google Cloud configuration for Vertex AI.

Parameters

  • project – Google Cloud project ID

  • location – Google Cloud location (e.g., “us-central1”)

convokit.genai.factory.get_llm_client(provider: str, config_manager, **kwargs)

Factory function as a unified interface to create LLM client instances.

Creates and returns the appropriate LLM client based on the provider name. The client is initialized with the config manager and any additional parameters.

Parameters

  • provider – Name of the LLM provider (“gpt”, “gemini”, “local”)

  • config_manager – Configuration manager instance to pass to the client

  • **kwargs –

    Additional parameters to pass to the client constructor

Returns

Initialized LLM client instance

Raises

ValueError – If the provider is not supported or dependencies are missing

LLMPromptTransformer

The LLMPromptTransformer is a flexible transformer that allows you to apply custom prompts and formatters to any level of corpus objects (utterances, conversations, speakers, or the entire corpus). It provides fine-grained control over how objects are formatted for LLM processing and where the results are stored.

Provider Clients

Supported Providers

Currently supported LLM providers:

  • OpenAI GPT: Access to OpenAI GPT models through the OpenAI API. See OpenAI API setup.

  • Google Gemini: Access to Google Gemini models via Vertex AI. See Vertex AI setup guide.

  • Local Models: Template implementation for local LLM models (requires custom implementation)

GPT Client

class convokit.genai.gpt_client.GPTClient(model: str = 'gpt-4o-mini', config_manager: convokit.genai.genai_config.GenAIConfigManager = None)

Client for interacting with OpenAI GPT models.

Provides an interface to generate text using OpenAI’s GPT models through their API. Handles authentication, request formatting, and error retry logic. API key is managed through the GenAI config system.

Parameters

  • model – Name of the GPT model to use (default: “gpt-4o-mini”)

  • config_manager – GenAIConfigManager instance (optional, will create one if not provided)

generate(prompt, output_max_tokens=512, temperature=0.0, times_retried=0) → convokit.genai.base.LLMResponse

Generate text using the GPT model.

Sends a prompt to the GPT model and returns the generated response. Handles different input formats (string or message list) and includes retry logic for API errors.

Parameters

  • prompt – Input prompt for generation. Can be a string or list of message dicts

  • output_max_tokens – Maximum number of tokens to generate (default: 512)

  • temperature – Sampling temperature for generation (default: 0.0)

  • times_retried – Number of retry attempts made so far (for internal use)

Returns

LLMResponse object containing the generated text and metadata

Raises

Exception – If output error and retry attempts are exhausted

Gemini Client

class convokit.genai.gemini_client.GeminiClient(model: str = 'gemini-2.0-flash-001', config_manager: convokit.genai.genai_config.GenAIConfigManager = None)

Client for interacting with Google Gemini models via Vertex AI.

This client is configured to use Vertex AI and requires Google Cloud project and location to be set. Configuration can be provided via the GenAI config system or environment variables.

Parameters

  • model – Name of the Gemini model to use (default: “gemini-2.0-flash-001”)

  • config_manager – GenAIConfigManager instance (optional, will create one if not provided)

generate(prompt, temperature=0.0, times_retried=0) → convokit.genai.base.LLMResponse

Generate text using the Gemini model.

Sends a prompt to the Gemini model and returns the generated response. The function includes retry logic for API errors and handles different input formats.

Parameters

  • prompt – Input prompt for generation

  • temperature – Sampling temperature for generation (default: 0.0)

  • times_retried – Number of retry attempts made so far (for internal use)

Returns

LLMResponse object containing the generated text and metadata

Raises

Exception – If retry attempts are exhausted

Local Client

The LocalClient provides a template implementation for integrating local LLM models. The current implementation returns mock responses and serves as a starting point for implementing actual local model support.

class convokit.genai.local_client.LocalClient(model_path: str = './', config_manager: convokit.genai.genai_config.GenAIConfigManager = None)

Template client for local LLM models. This is not a implemented client.

This is a template implementation for local LLM clients. It provides a mock implementation that should be replaced with actual local model loading and inference. Currently returns mock responses for testing purposes.

Parameters

  • model_path – Path to the local model files (e.g., llama.cpp or GGUF model)

  • config_manager – GenAIConfigManager instance (optional, will create one if not provided)

generate(messages, **kwargs) → convokit.genai.base.LLMResponse

Generate text using the local model.

Currently returns a mock response. This method should be implemented to actually load and run the local model for text generation.

Parameters

  • messages – Input messages for generation

  • **kwargs –

    Additional generation parameters

Returns

LLMResponse object containing the generated text and metadata

Adding New Providers

To add support for a new LLM provider:

  1. Create a new client class that inherits from LLMClient

  2. Update the configuration manager to support the new provider

  3. Implement the required generate() method and optionally stream() method if applicable

  4. Add the provider to the factory function in factory.py

Configuration

The GenAIConfigManager handles API key storage and retrieval for different LLM providers. It supports:

  • File-based storage: Configuration is stored in ~/.convokit/config.yml

  • Environment variables: API keys can be set via environment variables (e.g., GPT_API_KEY)

  • Secure storage: API keys are stored locally and not exposed in code

  • Provider-specific settings: Support for different configuration requirements per provider (e.g., Google Cloud project settings for Gemini)

Basic Usage:

from convokit.genai.genai_config import GenAIConfigManager

config = GenAIConfigManager()

# Set OpenAI API key
config.set_api_key("gpt", "your-openai-api-key")

# Set Google Cloud configuration for Gemini
config.set_google_cloud_config("your-project-id", "your-location")

# Configuration is automatically saved and can be reused