mcp_server_webcrawl.interactive package

Submodules

mcp_server_webcrawl.interactive.highlights module

class HighlightSpan[source]

Bases: object

Represents a highlight span in text

start: int
end: int
text: str
__init__(start, end, text)
Parameters:

  • start (int) –

  • end (int) –

  • text (str) –

Return type:

None

class HighlightProcessor[source]

Bases: object

Shared highlight processing utilities

QUOTED_PHRASE_PATTERN = re.compile('"([^"]+)"')
WORD_PATTERN = re.compile('\\b\\w+\\b')
SNIPPET_MARKER_PATTERN = re.compile("\\*\\*([a-zA-Z\\-_' ]+)\\*\\*")
IGNORE_WORDS = {'AND', 'NOT', 'OR', 'and', 'id', 'not', 'or', 'size', 'status', 'type', 'url'}
static extract_search_terms(query)[source]

Extract search terms from query, handling quoted phrases and individual keywords.

Parameters:

query (str) –

Return type:

List[str]

static find_highlights_in_text(text, search_terms)[source]

Find all highlight spans in text for the given search terms.

Parameters:
Return type:

List[HighlightSpan]

static extract_snippet_highlights(snippet_text)[source]

Extract highlights from snippet text with markers, returning clean text and highlights.

Parameters:

snippet_text (str) –

Return type:

tuple[str, List[HighlightSpan]]

static merge_overlapping_highlights(highlights, text)[source]

Merge overlapping or adjacent highlight spans.

Parameters:
Return type:

List[HighlightSpan]

static render_text_with_highlights(stdscr, text, highlights, x, y, max_width, normal_style, hit_style)[source]

Render text with highlights applied.

Parameters:
Return type:

None

mcp_server_webcrawl.interactive.search module

class SearchManager[source]

Bases: object

Manages search operations including async search and debouncing. Works with session’s controlled interface - never touches private state directly.

__init__(session)[source]
Parameters:

session (InteractiveSession) –

autosearch(immediate=False)[source]

Trigger search with optional immediate execution.

Parameters:

immediate (bool) – If True, execute search synchronously without debouncing. If False, use debounced async execution (default).

Return type:

None

cancel_pending()[source]

Cancel any pending search timer.

Return type:

None

check_pending()[source]

Check if there are pending search results and update the UI.

Return type:

None

cleanup()[source]

Clean up any pending operations.

Return type:

None

has_pending()[source]

Check if there’s a pending debounced search.

Return type:

bool

is_searching()[source]

Check if a search is currently in progress or on a timer.

Return type:

bool

mcp_server_webcrawl.interactive.session module

class InteractiveSession[source]

Bases: object

Main session coordinator that manages the interactive terminal application.

Initialize the interactive session with crawler and data source.

__init__(crawler, datasrc)[source]

Initialize the interactive session with crawler and data source.

Parameters:

  • crawler (str) –

  • datasrc (str) –

property ui_state: UiState
property ui_focused: UiFocusable
property crawler: BaseCrawler
property document: SearchDocumentView
property results: SearchResultsView
property searchform: SearchFormView
property searchman: SearchManager
property sites: list[SiteResult]
debug_add(msg)[source]

Add line of debug.

Parameters:

msg (str) –

Return type:

None

debug_clear()[source]

Clear debug statements.

Return type:

None

run()[source]

Public interface to launch the interactive terminal application.

Return type:

None

set_ui_state(state, focus=None)[source]

Transition between UI states cleanly.

Parameters:
Return type:

None

set_init_input_args(crawler, datasrc)[source]
Parameters:

  • crawler (str) –

  • datasrc (str) –

Return type:

None

set_init_crawler(crawler)[source]
Parameters:

crawler (BaseCrawler) –

Return type:

None

set_init_sites(sites)[source]
Parameters:

sites (str) –

Return type:

None

set_init_searchform(searchform)[source]
Parameters:

searchform (BaseCursesView) –

Return type:

None

get_theme_color_pair(theme)[source]
Parameters:

theme (ThemeDefinition) –

Return type:

int | None

mcp_server_webcrawl.interactive.ui module

class DocumentMode[source]

Bases: Enum

An enumeration.

MARKDOWN = 1
RAW = 2
HEADERS = 3
class NavigationDirection[source]

Bases: Enum

An enumeration.

UP = 1
DOWN = 2
LEFT = 3
RIGHT = 4
class ScrollDirection[source]

Bases: Enum

An enumeration.

UP = 1
DOWN = 2
class SearchFilterType[source]

Bases: Enum

An enumeration.

ANY = 0
PAGES = 1
class ThemeDefinition[source]

Bases: Enum

An enumeration.

DOCUMENT_MODE = (1, 1, 51)
HEADER_ACTIVE = (2, 7, 17)
HEADER_INACTIVE = (3, 7, 233)
HEADER_OUTER = (4, 7, 235)
HTTP_ERROR = (6, 7, 88)
HTTP_WARN = (7, 7, 130)
INACTIVE_QUERY = (8, 245, 237)
SNIPPET_DEFAULT = (9, 243, 0)
SNIPPET_HIGHLIGHT = (10, 232, 51)
UI_ERROR = (11, 7, 88)
class UiFocusable[source]

Bases: Enum

An enumeration.

UNDEFINED = 1
SEARCH_FORM = 2
SEARCH_RESULTS = 3
class UiState[source]

Bases: Enum

An enumeration.

UNDEFINED = 1
REQUIREMENTS = 2
SEARCH_INIT = 3
SEARCH_RESULTS = 4
DOCUMENT = 5
HELP = 6
safe_addstr(stdscr, y, x, text, style=0)[source]

Safe addstr that handles screen edge errors.

Parameters:

  • stdscr (window) –

  • y (int) –

  • x (int) –

  • text (str) –

  • style (int) –

Return type:

None

class InputRadio[source]

Bases: object

Radio input with 2-3 possible states (e.g., on/off or state1/state2/off)

Parameters:

  • group – The InputRadioGroup this radio belongs to

  • name – The form radio group name

  • label – The form radio label

  • index – The current state index

  • states – List of InputRadioState objects defining each possible state

__init__(group, name, label, index, states=None)[source]

Radio input with 2-3 possible states (e.g., on/off or state1/state2/off)

Parameters:

  • group – The InputRadioGroup this radio belongs to

  • name (str) – The form radio group name

  • label (str) – The form radio label

  • index (int) – The current state index

  • states (list | None) – List of InputRadioState objects defining each possible state

property current_state

Get the current state

property display_label: str

Get the current display label

property value: str

Get the current value

next_state()[source]

Cycle to the next state

Return type:

None

render(stdscr, y, x, field_index, max_width=None, focused=False)[source]

Render a single radio option.

Parameters:

  • stdscr (window) –

  • y (int) –

  • x (int) –

  • field_index (int) –

  • max_width (int | None) –

  • focused (bool) –

Return type:

None

set_state(index)[source]

Set the current state by index

Parameters:

index (int) –

Return type:

None

set_states(states)[source]

Set the current state by index

Parameters:

states (list) –

Return type:

None

class InputRadioGroup[source]

Bases: object

Radio group with navigation and layout management capabilities.

Radio input group with layout and navigation support.

Parameters:

  • name – The form radio group name (“filter”, “sort”, “site”, or “crawler”)

  • sites – List of SiteResult objects, required only for “site” group type

__init__(name, sites=None)[source]

Radio input group with layout and navigation support.

Parameters:

  • name (str) – The form radio group name (“filter”, “sort”, “site”, or “crawler”)

  • sites (list | None) – List of SiteResult objects, required only for “site” group type

property value: str
calculate_group_width()[source]

Calculate the display width needed for a radio group.

Return type:

int

clear()[source]
Return type:

None

set_layout_constraints(available_width, available_height, is_constrained=False)[source]

Set layout constraints for grid-based groups (like sites).

Parameters:

  • available_width (int) – Available horizontal space

  • available_height (int) – Available vertical space

  • is_constrained (bool) – Whether layout is constrained (affects sites per column)

Return type:

None

get_grid_position(radio_index)[source]

Convert linear radio index to grid position. Only applies to site groups; other groups return (radio_index, 0).

Parameters:

radio_index (int) – Linear index in radios list

Returns:

(row, column) position in grid layout

Return type:

tuple

get_index_from_grid(row, col)[source]

Convert grid position to linear radio index. Only works for site groups; returns None for other group types.

Parameters:

  • row (int) – Row in grid (0-based)

  • col (int) – Column in grid (0-based)

Returns:

Linear index if position exists within grid bounds, None otherwise

Return type:

int | None

navigate_left(current_radio_index)[source]

Navigate left within this group’s layout.

Parameters:

current_radio_index (int) – Current position in radios list

Returns:

New radio index if navigation successful, None if should exit group

Return type:

int | None

navigate_right(current_radio_index)[source]

Navigate right within this group’s layout.

Parameters:

current_radio_index (int) – Current position in radios list

Returns:

New radio index if navigation successful, None if should exit group

Return type:

int | None

navigate_to_row(target_row, from_column=0)[source]

Navigate to a specific row from an external column position.

Parameters:

  • target_row (int) –

  • from_column (int) –

Return type:

int | None

get_row_from_index(radio_index)[source]

Get the row number for navigation between groups.

Parameters:

radio_index (int) – Linear index in radios list

Returns:

Row number for inter-group navigation

Return type:

int

class InputRadioState[source]

Bases: NamedTuple

InputRadioState(label, value)

Create new instance of InputRadioState(label, value)

label: str

Alias for field number 0

value: str

Alias for field number 1

static __new__(_cls, label, value)

Create new instance of InputRadioState(label, value)

Parameters:

  • label (str) –

  • value (str) –

class InputText[source]

Bases: object

A reusable text input field with cursor management, rendering, and input handling. Consolidates the common text input functionality used across the application.

Initialize the text input field.

Parameters:

  • initial_value – Starting text value

  • max_length – Maximum allowed text length (None for unlimited)

  • label – Display label for the field

__init__(initial_value='', max_length=None, label='')[source]

Initialize the text input field.

Parameters:

  • initial_value (str) – Starting text value

  • max_length (int | None) – Maximum allowed text length (None for unlimited)

  • label (str) – Display label for the field

backspace()[source]

Remove the character before the cursor.

Return type:

None

clear()[source]

Clear all text and reset cursor.

Return type:

None

delete()[source]

Remove the character at the cursor position.

Return type:

None

end()[source]

Move cursor to the end of the text.

Return type:

None

handle_input(key)[source]

Handle keyboard input for the text field.

Parameters:

key (int) – The curses key code

Returns:

True if the input was handled, False otherwise

Return type:

bool

home()[source]

Move cursor to the beginning of the text.

Return type:

None

insert_char(char)[source]

Insert a character at the current cursor position.

Parameters:

char (str) –

Return type:

None

is_empty()[source]

Check if the text field is empty.

Return type:

bool

move_cursor_left()[source]

Move cursor one position to the left.

Return type:

None

move_cursor_right()[source]

Move cursor one position to the right.

Return type:

None

render(stdscr, y, x, width, focused=False, style=None)[source]

Render the text input field with box, text, and cursor.

Parameters:

  • stdscr (window) – The curses window

  • y (int) – Y position to render at

  • x (int) – X position to render at

  • width (int) – Total width of the input box

  • focused (bool) – Whether this field has focus (shows cursor)

  • style (int | None) – Curses style attributes to apply

Return type:

None

set_value(new_value)[source]

Set the text value and adjust cursor if needed.

Parameters:

new_value (str) –

Return type:

None

class ViewBounds[source]

Bases: object

__init__(x=0, y=0, width=0, height=0)[source]
Parameters:

Module contents