CIVET

CIF Validation and Editing Tool - A modern CIF (Crystallographic Information File) editor and validator with intelligent field checking, format conversion, and UTF-8 support.

Disclaimer
The code in this project has been written in large parts by Anthropic LLM models (mainly Sonnet 4.5 and Opus 4.5).

Be advised that this software is in constant development and might therefore contain bugs or other unintended behaviour. Always check your CIF files carefully and if you encounter an issue and would like to report it, please do so via the Issues section.

Known checkCIF Compatibility Notes

A few CIVET behaviours exist specifically to keep files compatible for example with IUCr's checkCIF service:

• Multiline text blocks are never rewritten. Content inside a semicolon-delimited (;...;) or CIF 2.0 triple-quoted ("""...""" / '''...''') value — such as the reflection/refinement narrative in _iucr_refine_fcf_details — is treated as opaque text. CIVET does not convert notation, flag deprecated data names, or otherwise modify anything inside these blocks, even if the text happens to contain data-name-like strings (e.g. software echoing part of the CIF back into a details block). This is intentional: checkCIF and other downstream tools may expect that content to survive untouched. I will review this periodically and change the behaviour if it becomes safe to do so.
• Some legacy data names are deliberately retained alongside their modern successors. For example, _cell_measurement_temperature is kept even when the modern _diffrn_ambient_temperature is present, because checkCIF's PLAT197 check does not yet recognise the modern name. The full list of these data names (as far as I am aware) lives in field_rules/checkcif_compatibility.cif_rules and is reviewed periodically as checkCIF is updated.

Key Features (v1.3)

• CIF Editing and Validation: Syntax highlighting, guided dialogs, smart field checks, and duplicate/alias-aware workflows.
• Command-Line File Opening: Launch CIVET with an optional CIF file path argument to open it on startup (unknown arguments are ignored, so Qt flags can be passed through safely).
• Flexible Field Rules Engine: Supports CHECK, DELETE, EDIT, RENAME, CALCULATE, and APPEND actions in .cif_rules files.
• Legacy/Modern CIF Handling: Detects legacy, modern, and mixed notation; includes conversion plus integrated malformed-field detection and correction.
• .cif_rules Notation Converter: Converts a field-rules file between legacy and modern notation from the main menu; when a loaded CIF's notation doesn't match the active rule set, Start Checks offers to convert the rules, convert the CIF, run as-is, or cancel.
• CIF Syntax Compliance Dialog: Dedicated tabbed dialog (CIF 2.0 / CIF 1.1 / all) for syntax-version compliance issues, with line navigation, scoped Fix All actions, refresh, and a non-ASCII character conversion entry point.
• CIF2 Compliance Support: Maintains #\#CIF_2.0 headers and handles CIF2 quoting/formatting edge cases (including triple-quoted values and CIF2 lists/tables).
• Dictionary-Backed Intelligence: Multi-dictionary loading, metadata display, update checks, and parser support for DDLm + DDL1 dictionaries.
• Dictionary Search: Search loaded dictionaries by data name, alias, category, and optionally description text; filter to selected dictionaries and cross-check hits against the currently loaded CIF.
• Data Name Validation: Validates names against loaded dictionaries and IUCr registered prefixes, groups malformed/unknown/deprecated names, offers one-click fixes (including Replace and Delete for deprecated fields with an existing successor), and uses validation-aware highlighting.
• Data-Name Integrity Resolution: Detects duplicate data names and alias groups with conflicting values, then offers guided manual or auto-resolution in save/conversion workflows.
• Data Value Validation: Validates field values against dictionary-defined types, numeric ranges, and enumeration sets; detects loop count mismatches. Results shown in a sortable, live-refreshable dialog. Accessible via Actions → Validate Data Values...
• Live File Status Panel: Side-by-side with field-rule selection, a compact status panel tracks syntax compliance (CIF 2.0 / CIF 1.1 / not compliant), notation state (modern/legacy/mixed), and latest data-name/data-value validation outcomes.
• Persistent User Configuration: Cross-platform storage for settings, user rules, recognised prefixes, and downloaded dictionaries.
• Productivity UX: Built-in/user/custom rules selection, dropdown suggestions for field values, configurable dialog interaction modes, and focused editor settings.
• Performance: Debounced/background compliance checks, content-hash-based reparse avoidance, and caching across dictionary lookups and validation keep the editor responsive on larger files.

Quick Start

From Source (Windows)

git clone https://github.com/danielnrainer/CIVET.git
cd CIVET
pip install -r requirements.txt
python src/main.py

Optionally pass a CIF file to open on startup: python src/main.py path/to/file.cif.

Standalone Executable (Windows)

1. Download CIVET.exe from releases
2. Run directly - no Python installation required

From source (Linux)

Linux distributions do not like use of pip outside a virtual environment:

git clone https://github.com/danielnrainer/CIVET.git
cd CIVET
mkdir civet_virtual
python -m venv ./civet_virtual
source ./civet_virtual/bin/activate
pip install -r requirements.txt
python src/main.py

If the error message From 6.5.0, xcb-cursor0 or libxcb-cursor0 is needed to load the Qt xcb platform plugin appears when you run python src/main.py, you need to install packages libxcb-cursor0 and libxcb-util1 or equivalent package names for your distribution.

To execute after the initial installation:

source CIVET/civet_virtual/bin/activate
python CIVET/src/main.py

Building Executable

pip install -r requirements-dev.txt
pyinstaller CIVET.spec

Compiled releases bundle only the runtime application and required runtime dependencies; development and test tools are not included.

Custom Field Rules

Create validation rules using .cif_rules files:

# Example custom field rules
_chemical_formula_sum         ?        # Chemical formula
_space_group_name_H-M_alt     'P 1'    # Space group
_diffrn_ambient_temperature   293      # Temperature in K

# Special actions:
DELETE: _field_to_remove           # Removes field from CIF
EDIT: _field_name new_value        # Replaces field value
RENAME: _old_name _new_name        # Renames field (for correcting erroneous field names)
CALCULATE: _field = expression     # Calculates value from other fields
APPEND: _publ_section_references Allen, F.H. (2010), Acta Cryst B66, 380-386.  # Appends to multiline field
    Multiple APPEND entries (in the same .cif_rules file) for the same CIF field (data name) will be concatenated

# CALCULATE example - convert fluence to flux density:
# CrysAlisPRO reports fluence (e/Ų) as _diffrn.flux_density, but flux density is e/Ų/s
CALCULATE: _diffrn.flux_density = _diffrn.flux_density / (_diffrn.total_exposure_time * 60) 

Built-in sets include packaged .cif_rules files from field_rules/ (for example 3DED modern and 3DED legacy).

User Custom Field Rules (AppData)

CIVET supports persistent user-created field rules stored in your system's application data directory:

Location:

• Windows: %APPDATA%\CIVET\field_rules\ (typically C:\Users\[username]\AppData\Roaming\CIVET\field_rules\)
• macOS: ~/Library/Application Support/CIVET/field_rules/
• Linux: ~/.config/CIVET/field_rules/

Field Rules Selection UI: The application provides three ways to select field rules:

1. Built-in: Select from field rules that ship with CIVET (3D ED Modern, 3D ED Legacy, High Pressure)
2. User: Select from your custom rules stored in the AppData directory (dropdown shows available files)
3. Custom File: Browse to select any .cif_rules file from your filesystem

Quick Access:

• Click the 📁 button next to the User dropdown to open the user rules directory
• Click the ↻ button to refresh the list after adding new files
• Use Settings → Open User Config Directory... from the menu

User Data Directory

CIVET stores all user configuration and customizations in a platform-specific application data directory:

Platform	Location
Windows	%APPDATA%\CIVET\
macOS	~/Library/Application Support/CIVET/
Linux	~/.config/CIVET/

Contents:

CIVET/
├── settings.json           # Editor preferences (font, ruler, etc.)
├── registered_prefixes.json  # Custom CIF prefix registry
├── dictionaries/           # User-downloaded CIF dictionaries
└── field_rules/            # Custom validation rules

This allows settings and customizations to persist across sessions and software updates, even when using the standalone executable.

Editor Settings

CIVET provides customizable editor preferences accessible via Settings → Editor Settings...:

• Font Family: Choose from system-installed fonts (default: Courier New)
• Font Size: Adjustable size in points (default: 10)
• Line Numbers: Toggle line number display
• Syntax Highlighting: Enable/disable CIF-specific syntax coloring
• Syntax Highlighting Colors: Adjust the highlight colours used for valid, unknown, malformed, deprecated, loop, and value text directly in the dialog
• 80-Character Ruler: Visual guide for line length
• Dialog Behavior: Three interaction modes for result dialogs: Allow editing while open (non-blocking, default for Validate Data Values), Browse editor (read-only) while open, and Classic modal (lock editor). Each dialog can inherit the global default or set its own mode.

All settings are stored in settings.json and persist across sessions. User settings always take precedence over built-in defaults. Use Reset to Defaults to restore original settings.

Advanced users can also customise syntax-highlighting colours manually in settings.json under editor.syntax_highlighting_colors.

Data Value Validation

The Actions → Validate Data Values... dialog checks all field values in the current file against dictionary definitions:

• Type mismatches: flags non-numeric values for Real/Integer fields
• Enum violations: flags values not listed in _enumeration_set.state
• Loop count errors: flags loops where the number of data values is not a multiple of the field count
• Results are shown in a sortable table with severity colour-coding (error/warning/info)
• Refresh: re-runs validation against the current editor content without closing the dialog, useful when editing to fix issues
• Go to Line: navigates the editor to the flagged line (double-click also works)

The dialog opens in non-blocking mode by default so you can edit while it is open. This can be changed under Settings → Editor Settings → Dialog Behavior.

Dictionary Search

The Dictionaries → Search Loaded Dictionaries... dialog provides a searchable view over all currently loaded dictionaries:

• Searches canonical data names, known aliases, categories, and optionally description text
• Supports multi-dictionary selection, with the active CORE dictionary selected by default when available
• Marks whether hits are present in the current CIF, including separate states for canonical names, alias-only matches, and cases where both forms are present
• Shows dictionary metadata, known aliases, units, examples or allowed values, and the first matching CIF line
• Lets you jump directly to the matching line in the editor

You can also select a data name in the editor, right-click, and use Search in Dictionaries to open the dialog prefilled with that text.

Data Name Validation Results

The validation results dialog now combines several related checks in one place:

• Malformed Fields: Detects names that are structurally wrong (including misplaced dots in modern notation) but can be mapped to a known field, and offers one-click, notation-aware correction.
• Unknown Fields: Flags names not recognised by loaded dictionaries or registered prefixes.
• Deprecated Fields: Offers a + Successor action that adds the appropriate successor name while keeping the deprecated field, a Replace action that swaps the deprecated name for its successor in place, and a Delete action to remove the deprecated field once its successor already exists. In legacy or mixed CIF files, CIVET prefers a legacy successor name when one exists; otherwise it falls back to the modern name.
• Conflict Resolution Controls: The conflict dialog now supports keeping aliases while synchronizing all alias values when that strategy is preferred.
• Details Tooltips: Hover over the Details column to read the full text when the column is truncated.

Malformed-field fixes are now handled through the validation dialog itself rather than through a separate pre-check option.

Syntax Highlighting Categories

When dictionary-aware validation highlighting is enabled, CIVET distinguishes field categories directly in the editor:

• Valid fields: green
• Modern-only fields: strong blue with underline (known modern names that have no legacy notation alias)
• Registered local-prefix fields: cyan/teal
• User-allowed fields: cyan/teal italics
• Unknown fields: red
• Malformed fields: red italics
• Deprecated fields: dark yellow with strikethrough

Without validation-aware categorisation, data names use a default purple fallback. Quoted values and multiline values are blue, loop_ is bold orange, loop field names inherit their category styling and are italicised, and loop data values use a darker orange.

If you are new to the colour scheme, use Help → Syntax Highlighting Guide... in the application for a quick explanation of what each colour and text style means. The guide reflects the currently active colours, including any custom values you set in settings.json.

Advanced users can also customise syntax-highlighting colours manually in settings.json under editor.syntax_highlighting_colors, including modern_only.

Data-Name Integrity Checks

To prevent conflicting aliases and duplicate data names from being written accidentally, CIVET now enforces data-name integrity in key workflows:

• Save blocks when unresolved duplicate/alias-value conflicts remain.
• Conversion and automated fix operations prompt to resolve conflicts when detected.
• Auto-resolution keeps a single recommended field by default; manual resolution can preserve aliases and synchronize values.

Editing Convenience

• Reload File: Available from Edit → Reload File with Ctrl+Shift+R, this reloads the current file from disk and discards unsaved editor changes after confirmation.

Building

Executable

pip install pyinstaller
pyinstaller CIVET.spec

Output: dist/CIVET (or CIVET.exe on Windows, CIVET.app on macOS)

System Requirements

• Executable: Windows 10/11, macOS 10.14+ (not tested), Linux (not tested)
• Source: Python 3.11+, PyQt6, requests

License

BSD Clause License - see LICENSE

Citation

CIVET - CIF Validation and Editing Tool
GitHub: https://github.com/danielnrainer/CIVET
Zenodo: https://doi.org/10.5281/zenodo.17328490