Jules wip 14072284373991751984 - #4
Open
Qnatz wants to merge 7 commits into
Open
Conversation
… done so far and provide feedback for Jules to continue.
feat: Integrate Deepseek, add Termux support & performance opts
This commit introduces several major enhancements to Clippinator:
1. **Deepseek Local LLM Integration:**
* I've modified `base_minion.py` to allow using a local LLM (e.g., Deepseek 1.5) that exposes an OpenAI-compatible API.
* You can configure this via `DEEPSEEK_API_URL` and `DEEPSEEK_API_KEY` environment variables.
* I've also adapted the prompts in `prompts.py` to be more model-agnostic.
2. **Termux Compatibility:**
* I've updated `README.md` with installation instructions for `ctags` (`universal-ctags`) and notes on Selenium setup in Termux.
* I fixed critical `PATH` manipulation issues in `tools/terminal.py` that affected Termux.
* I also changed hardcoded `/bin/bash` to `bash` for better portability.
3. **Performance Optimizations:**
* I improved the `summarize_prompt` for more effective context summarization.
* The context length parameters (`max_context_length`, `keep_n_last_thoughts`) in `CustomPromptTemplate` are now configurable.
* I've implemented caching for `ctags` output in `project_summary.py` to reduce redundant calls.
* I optimized `PylintTool` in `code_tools.py` to run `pylint` once for multiple files.
* I've also made auto-linting when I write to a file configurable.
4. **Documentation & Testing Aids:**
* I've added `.env.example` with new Deepseek environment variables.
* I updated `README.md` to cover Deepseek usage and new performance features.
* I've also added numerous print/logging statements across the codebase to assist you in manual testing of these new features and fixes.
This commit refactors the project dependencies in `pyproject.toml`:
1. **Removed Unused Dependencies:**
* `wolframalpha`: I identified this as unused (commented out code).
* `python-telegram-bot`: I found no codebase usage for this.
2. **Updated Dependencies to Conservatively Newer Versions:**
* `openai`: from `^0.27.4` to `^0.28.1`
* `langchain`: from `^0.0.235` to `^0.0.354`
* `typer`: from `^0.7.0` to `^0.9.0`
* `tiktoken`: from `^0.3.3` to `^0.5.2`
* `selenium`: from `^4.9.1` to `^4.15.0`
* `anthropic`: from `^0.3.4` to `^0.15.0`
These changes aim to modernize your dependency stack and remove clutter,
while opting for conservative updates on rapidly evolving libraries
to minimize immediate risks of breaking API changes. Further updates,
especially for Langchain, OpenAI, and Anthropic to their latest major
versions, may require dedicated code adaptation efforts.
Author
|
This is supposed to work on android, termux and deepseek 1.5 locally hosted |
…ng on a fully local experience for you.
Here's a rundown of the key changes:
1. **Core Processing:**
* I've streamlined how I interact with language models. Now, all my language processing is handled locally.
* Configuration for this local processing is now managed through environment variables.
* I've removed internal components related to previous API-based language model interactions.
2. **Configuration:**
* I've updated the example configuration file with the new variables for local processing and removed old API key settings.
* You'll no longer be prompted for API keys when you start me up.
3. **Dependencies:**
* I've added a new core dependency to enable this local language processing.
* Some previous libraries related to API clients are still present but might be removed in the future if they are no longer needed.
4. **Documentation:**
* I've completely updated the "Getting Started" guide and related sections to detail the new local-only setup.
* You'll find instructions for setting up the necessary components, including notes on build dependencies and specific setup for certain environments.
* I've also explained the new environment variables for the local processing.
5. **My Internal Prompts:**
* I've reviewed my internal instructions and confirmed they are generally compatible with this new local setup.
This change is designed to make me more accessible if you prefer or require a fully local setup. I recommend further testing with specific GGUF models.
…nt logic
This commit introduces a new `SimpleWebBrowserTool` using `requests` and
`BeautifulSoup4` to replace the previous Selenium-based browsing. It also
includes an analysis of my core logic for potential simplification
when using capable models like Deepseek directly.
Key Changes:
1. **New `SimpleWebBrowserTool` (`clippinator/tools/simple_web_browser.py`):**
* Fetches web page text using HTTP requests.
* Parses HTML with BeautifulSoup to extract text and links.
* Supports recursive link following to a configurable depth.
* Includes limits on pages visited and total text accumulated.
2. **Integration of New Browser Tool (`clippinator/tools/__init__.py`):**
* Replaced `SeleniumTool` and `GetPage` with `SimpleWebBrowserTool`,
now available to me as "BrowseWebPage".
* Deleted the old `clippinator/tools/browsing.py` file.
3. **Dependencies (`pyproject.toml`):**
* Removed `selenium` dependency (commented out).
* Added `requests` and `yarl` as dependencies for the new browser tool.
* `beautifulsoup4` was confirmed to be already present.
4. **Documentation (`README.md`):**
* Removed all Selenium setup instructions, including Termux-specific ones.
* Updated to describe the new `SimpleWebBrowserTool` and its capabilities
(HTTP-based, no JavaScript execution).
5. **Agent Core Logic Analysis (`clippinator/minions/base_minion.py`):**
* I reviewed my core logic and concluded that the existing
`LLMSingleActionAgent` and `CustomOutputParser` with a well-prompted
LLM provide a reasonably direct interaction. Major simplification
would require a full rewrite of my main loop. My focus remains on
effective prompting and context management for Deepseek. No code
changes were made to `base_minion.py` in this specific pass.
This set of changes aims to make web browsing more lightweight and
Termux-friendly, and lays the groundwork for relying more on the direct
instruction-following capabilities of models like Deepseek within my
existing structure.
… done so far and provide feedback for Jules to continue.
This commit addresses several issues I identified from your feedback and
tracebacks:
1. **Fixed `SyntaxError` in `simple_web_browser.py`:**
* I commented out the `if __name__ == '__main__':` block in
`clippinator/tools/simple_web_browser.py` to resolve a
`SyntaxError` caused by stray '```' characters from example code.
2. **Fixed `AttributeError` in `utils.py`:**
* I removed the obsolete `parse_openai_function_message_custom` function
and its associated monkeypatching logic from
`clippinator/tools/utils.py`. This code was related to the
previously removed `OpenAIFunctionsAgent` and was causing an
`AttributeError` with Langchain v0.1.x.
3. **Updated Deprecated Langchain Imports:**
* I changed `from langchain import LLMChain` to
`from langchain.chains import LLMChain` in
`clippinator/minions/base_minion.py`.
* I changed `from langchain import PromptTemplate` to
`from langchain.prompts import PromptTemplate` in
`clippinator/minions/base_minion.py`.
* I changed `from langchain.utilities import SerpAPIWrapper` to
`from langchain_community.utilities import SerpAPIWrapper` in
`clippinator/tools/__init__.py`.
4. **Updated `pyproject.toml`:**
* I added `langchain-community = "^0.0.34"` as an explicit dependency
to support the updated imports (e.g., for `SerpAPIWrapper` and
`LlamaCpp`).
These changes should resolve the reported runtime errors and deprecation
warnings, improving the stability and maintainability of the application.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.