---

PoliPrompt

PoliPrompt is a high-performance, cost-effective LLM-based text classification framework designed for political science research. It automates prompt optimization and dynamically selects exemplars during few-shot learning inference to improve the accuracy of classification tasks, especially in the context of sentiment analysis, stance detection, and more.

Getting Started

A. Set Up Virtual Environment and Install PoliPrompt Package

1. Use the following command to display your current directory (user home directory):

pwd

2. Navigate to the Directory (e.g. user home directory) Where You Want to Set Up the Virtual Environment

cd ~

3. Run the following command to create a virtual environment named poliprompt_env (changeable):

python3 -m venv poliprompt_env

4. Activate the Virtual Environment

source poliprompt_env/bin/activate

5. After activating the environment, install the PoliPrompt package:

pip install PoliPrompt

6. Download the Poliprompt source code for examples
   • To run the example code, users need to download the source code for notebooks and examples
   • Visit project home page on github
   • Download the project by git clone HTTP/SSH in terminal or Download ZIP (please refer to github basic tutorials)
   • Put it in user specified location on local machine, for example, the user home directory ~.

B. Register and Obtain API Keys

To use PoliPrompt effectively, you will need API keys from various large language model platforms. Follow these steps to register and retrieve your keys:

1. Register on LLM Platforms:

   • Visit the links below to sign up for API access from popular large language model providers:
     • OpenAI – Sign up for GPT-based APIs.
     • Anthropic – Register for Claude-based APIs.
     • Voyage AI – Obtain access to additional AI tools.

2. Generate and SAVE API Keys:

   • Once registered, navigate to the API section of each platform's dashboard.
   • Generate new API keys.
   • Copy the keys for use in your .env file.

3. Create and Save the .env File:

   • In your terminal, navigate to the root directory of your project (e.g., ~/PoliPrompt).
   • Run the following command to open a new .env file for editing:

   vim .env

   • In the .env file, add your API keys in the following format:

   OPENAI_API_KEY=your-openai-api-key
   ANTHROPIC_API_KEY=your-anthropic-api-key
   VOYAGE_API_KEY=your-voyage-api-key

C. Run the Example Code

1. Install Jupyter Notebook

If Jupyter Notebook is not already installed, you can install it using pip:

pip install notebook

2. To use the PoliPrompt virtual environment in Jupyter, you need to create a new kernel:

pip install ipykernel
python -m ipykernel install --user --name=PoliPrompt_env --display-name "PoliPrompt Kernel"

3. Navigate to the folder where the TopicExperiment.ipynb file is located. For example, if the file is in the notebooks folder inside the PoliPrompt directory, run:

cd ~/PoliPrompt/notebooks

4. Start Jupyter Notebook by running the following command:

jupyter notebook

5. Select the PoliPrompt Kernel -Once the notebook is open: -Navigate to the top-right corner of the notebook interface. -Click on "Kernel" → "Change Kernel." -Select PoliPrompt Kernel from the dropdown list.

6. After selecting the kernel, you can run the example code (TopicExperiment.ipynb) inside the notebook.

Citation

To cite the PoliPrompt paper, please use the following BibTeX reference:

@article{liu2024poliprompt,
  title={PoliPrompt: A High-Performance Cost-Effective LLM-Based Text Classification Framework for Political Science},
  author={Liu, Menglin and Shi, Ge},
  journal={arXiv preprint arXiv:2409.01466},
  year={2024}
}