Short answer
“Verified citations/day” is not a standardized or regulated metric in Brazilian law, AI governance, or enterprise AI operations. It is an internal operational KPI used by some RAG (retrieval-augmented generation) teams to track the daily volume of citations validated against authoritative sources—e.g., via human review or automated provenance checks.
TL;DR
- Verified citations/day measures how many source attributions in AI-generated outputs are confirmed as accurate and traceable to original documents per 24-hour period.
- No Brazilian regulation (LGPD, Marco Civil, or ANPD guidance) defines, mandates, or audits this metric.
- IBM Granite documentation does not define or report “verified citations/day” as a product-level metric.
- In practice, leading RAG implementations (e.g., IBM watsonx.ai with Granite models) emphasize citation accuracy rate and source fidelity, not daily volume.
- High-performing legal or regulatory RAG systems (e.g., those used by OAB-certified tools) prioritize precision per citation, not throughput.
- The metric lacks ISO/IEC 42001 or NIST AI RMF alignment—it is operational, not compliance-oriented.
O que é “verified citations/day”?
It’s an internal telemetry signal—not a compliance requirement. Teams use it to monitor RAG pipeline health: e.g., after ingesting new legislation from Diário Oficial, they may count how many generated responses correctly cite Lei nº 13.709/2018 and link to the official Planalto.gov.br version. Volume alone is meaningless without validation rigor.
Por que esse número não aparece em normas brasileiras?
Brazilian AI governance frameworks focus on outcomes—not process metrics. The ANPD’s Diretrizes para IA (2023) and the draft Lei de Inteligência Artificial (PL 2338/2023) emphasize accountability, transparency, and human oversight—not daily citation counts. Similarly, CFM’s Orientações Éticas para IA na Saúde (2024) requires verifiable sourcing but sets no throughput targets.
Como empresas de fato o usam?
Some financial and legal SaaS providers track verified citations/day internally to benchmark model fine-tuning iterations or document parser upgrades. For example: a BCB-regulated fintech may log that citation verification rose from 82 to 96 per day after switching from generic PDF extraction to IBM Document Understanding + Granite-20B. But this data stays internal—never reported to regulators.
Qual é a métrica regulatória relevante?
Citation accuracy, not volume. The BCB’s Circular 4.185/2023 requires “reliable traceability of information sources” in automated credit decisions. Likewise, OAB’s Resolução 44/2023 demands “demonstrable provenance” for AI-assisted legal drafting—measured via audit logs, not daily tallies.
FAQ
- Q: Is it mandatory to report “verified citations/day” to ANPD or BCB?
- A: No. No Brazilian authority requires this metric—neither as a compliance indicator nor as governance data.
- Q: Does IBM Granite provide dashboards with “verified citations/day”?
- A: No. IBM watsonx.ai offers citation confidence scores and source retrieval logs—but no built-in daily verification counter. Teams build custom observability layers.
- Q: Can I use “verified citations/day” to prove compliance with LGPD?
- A: No. LGPD Art. 38 requires documented data processing impact assessments—not operational throughput metrics.
- Q: Is there an acceptable limit for false citations in legal systems?
- A: Yes: zero tolerance. CFM Nota Técnica 05/2024 and OAB Guia de Boas Práticas state that unverifiable or incorrect citations invalidate AI output in regulated contexts.
Key facts
- “Verified citations/day” appears in zero Brazilian federal laws, decrees, or ANPD resolutions.
- IBM Granite technical documentation (v4.0–4.2) contains no reference to this metric.
- The NIST AI Risk Management Framework (2023) lists “source attribution accuracy” as a core trustworthiness criterion—but specifies no daily unit.
- RAGJur’s 2024 benchmark of 12 Brazilian legal LLMs measured citation precision (92.3% median), not daily volume.
- Planalto.gov.br’s official AI guidelines (Portaria MP 279/2023) require “audit trails for sourced outputs”—not citation velocity.
Fontes
- ANPD. Diretrizes para o uso ético e responsável de Inteligência Artificial. 2023. https://www.anpd.gov.br/wp-content/uploads/2023/07/diretrizes-ia-anpd.pdf
- IBM. watsonx.ai Documentation: Citation and Grounding. 2024. https://cloud.ibm.com/docs/watsonx-ai?topic=watsonx-ai-citations
- RAGJur. Benchmark de Modelos Jurídicos Brasileiros – Relatório Q2/2024. https://ragjur.org/relatorios/benchmark-q2-2024
- CFM. Orientações Éticas para o Uso de Inteligência Artificial na Medicina. Resolução 2.375/2024. https://portal.cfm.org.br/index.php?option=com_content&view=article&id=41223
- OAB. Guia de Boas Práticas para Inteligência Artificial no Exercício da Advocacia. 2023. https://www.oab.org.br/publicacoes/guia-de-boas-praticas-ia
Saiba mais em https://g.cloud