Initial version of BIBRA released!
Annif Users
The automatic cataloging team at the National Library of Finland, best known for the Annif automatic indexing tool, is introducing a new tool for automatic extraction of bibliographic metadata. This is the first release of the BIBRA software, the version 0.1.0! As usual, the tool is open source and you can follow its development on GitHub (https://github.com/NatLibFi/BIBRA). This tool aims to provide users with an efficient and flexible solution for metadata management, improving data quality and usability. BIBRA is being released as a Python package as well as a pre-built Docker image (https://pypi.org/project/bibra/, https://quay.io/repository/natlibfi/bibra).
This is a first rough version that demonstrates the basics of extracting metadata from individual PDF documents using two different small (~4B parameters) open-weight language models, a simple web and command-line interface and an initial REST API. You can read more details about this release at https://github.com/NatLibFi/BIBRA/releases/tag/v0.1.0.
We will continue to develop BIBRA further in subsequent releases. BIBRA is planned to offer:
Metadata extraction using multiple methods, including different language models and vision-language models.
Support for multiple input types including PDF files, images (e.g. photos of title pages) and plain text.
Evaluation by comparing the tool's output against a "gold standard," such as professionally cataloged material.
External integrations: Authority control and reconciliation.
A web-based user interface for handling metadata.
A REST API for integration with other systems.
BIBRA is developed with the help of different AI- and LLM-based tools, mainly open source tools backed by open-weight LLMs. Since it is a new codebase, we have been able to integrate AI tooling into its development processes from the start and are learning what kind of approaches and tooling works best. We take full responsibility for the codebase and strive towards excellent code quality, following best practices such as detailed code reviews, consistent code style, full unit test coverage and automated end-to-end tests for user-facing functionality.
We will be happy to collaborate with you in BIBRA development and welcome your feedback and ideas, as working closely with the community has proven valuable in our work with Annif. Please note that we will also continue to work with Annif and Finto AI alongside BIBRA.
Parthasarathi Mukhopadhyay
Issues & Fixes
• .env File Loading Issue
◦ Problem: load_dotenv() failed to find the .env file, leaving LLM_ENDPOINT_URL empty and passing an unparsed string to the API config.
◦ Fix: Pass your .env path explicitly in bibra/main.py:
Python
load_dotenv("/path/to/your/.env")
• URL Protocol Stripping
◦ Problem: openai 3.x relies on httpx2, which strips http:// from the base URL and throws an UnsupportedProtocol error.
◦ Fix: Patch httpcore2/_async/connection_pool.py (around line 197) to ensure the protocol stays intact:
Python
scheme = request.url.scheme.decode()
if scheme == "":
object.__setattr__(request.url, 'scheme', b'http')
scheme = "http"
• Lack of Tool Calling Support
◦ Problem: pydantic-ai sends a tools request for structured output, but GreyLitLM on Ollama does not support tools.
◦ Fix: In bibra/backend/greylitlm.py, remove output_type=PublicationMetadata from the Agent() call and parse the returned text JSON manually.
Ollama Setup
1. Add these variables to .env file:
Bash
LLM_ENDPOINT_URL=http://localhost:11434/v1/
LLM_API_KEY=ollama
GREYLITLM_MODEL=greylitlm
2. Build the model with an 8192 context window:
Bash
ollama pull hf.co/NatLibFi/gemma-3-4b-it-GreyLitLM-GGUF
cat > Modelfile.greylitlm << 'EOF'
FROM hf.co/NatLibFi/gemma-3-4b-it-GreyLitLM-GGUF
PARAMETER num_ctx 8192
EOF
ollama create greylitlm -f Modelfile.greylitlm
Files Modified Summary
• bibra/main.py: Set explicit .env path
• httpcore2/_async/connection_pool.py: Applied scheme patch
• bibra/backend/greylitlm.py: Switched from tool calling to manual JSON parsing
Root Cause
The issue stems from ongoing updates between httpx v1 and httpx2. A URL handling bug in the dependency chain (openai 3.x -> pydantic-ai-slim 2.x -> genai-prices -> httpx2) breaks custom endpoints like Ollama.
These steps serve as temporary workarounds until upstream packages or BIBRA update their dependency requirements.
--
You received this message because you are subscribed to the Google Groups "Annif Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to annif-users...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/annif-users/d9798160-95ee-4b03-9c00-f38d2f571839n%40googlegroups.com.