Fine-tuning of transformer-based models to predict semantically relevant outlinks in large-scale web crawling. Includes binary classification, score estimation with QualT5, and a multimodal BERT architecture integrating textual and metadata features for robust frontier prioritisation.
information-retrieval web-crawling fine-tuning multimodal-deep-learning msmarco clueweb neural-ir t5 bert-fine-tuning huggingface-transformers transformers-models frontier-prioritisation
-
Updated
Sep 15, 2025 - Jupyter Notebook