Datadog

Datadog AI

Datadog AI

Datadog AI

Datadog AI

Datadog AI

Datadog AI

Datadog AI

Datadog AI

Datadog AI

Datadog AI

Cloud scheduled tasks can't access MCP connectors — anyone find a workaround or solution? Or have any insight on it beyond what I list here?

Scheduled tasks on Claude Code (cloud, via claude.ai/code/scheduled) can't see any MCP connectors when they fire autonomously. Doesn't matter which connector — I've tested with multiple Zoho connectors and Microsoft 365. The agent runs ToolSearch, finds nothing, and tells you the tools need to be connected. They're connected. They work fine in interactive chat. The tell: if you open the failed session and send any message — literally just "try again" — everything works instantly. No config changes. The tools just appear once a human is in the session. This makes scheduled tasks useless for anything that touches an external service. Email summaries, channel monitoring, CRM lookups, posting to chat platforms — none of it works autonomously. Which is the entire point of scheduling. What I've tried (nothing works): - Deleted and recreated the task - Disabled all connectors on the task, saved, re-enabled, saved - Simplified to a minimal test prompt - Switched models - Different prompt content entirely This SEEMS TO BE a known bug with no workaround. Multiple GitHub issues document it across different connectors (Slack, Datadog, Jira, Zoho, Chrome) and across both Desktop and cloud tasks: #35899 — connectors not available until user message warms session #36327 — same, closed as duplicate #32000 — missing auth scope in scheduled sessions #40835 — editing a task silently disables connectors No one has posted a workaround. No Anthropic team member has commented on any of these issues. I filed my own report since the existing ones are mostly from Desktop/Cowork users — I'm on Teams, cloud-only, no Desktop fallback: 👉 https://github.com/anthropics/claude-code/issues/43397 Anyone else dealing with this? Found anything that works? Workaround Found ! Reddit user /u/e_lizzle was able to identify a workaround that worked for me - it'll cost a few extra tokens but if you start the scheduled task prompt by telling it to not do any work but instead to use an agent to do the entire task - everything works fine because the subagent gets mcp tools initialized properly. Then for now I'm telling it to have the subagent report a summary back up to the primary so I can look at its results in the task log. Cost difference is probably negligible and it solves the problem until its formally fixed. submitted by /u/checkwithanthony [link] [comments]

You gotta admit that this is cool.

graph TD subgraph Entry["AREA-1: Entry & CLI Parsing"] CLI_TSX["src/entrypoints/cli.tsx\n--version fast-path (zero imports)\nABLATION_BASELINE feature gate"] MAIN["src/main.tsx 4684 LOC\nCommander.js parser\nParallel MDM+Keychain+API prefetch\n9 config migrations on startup"] INIT["src/entrypoints/init.ts\nmemoized init()\nOrder: configs→env→shutdown→1P-events\n→OAuth→JetBrains→remote-settings\n→policy-limits→mTLS→proxy→preconnect"] CMDREG["src/commands.ts 758 LOC\n100+ slash commands\nStatic + feature-gated + ant-only\n+ skills + plugins (dynamic)"] REPL_L["src/replLauncher.tsx\nlaunchRepl() → App+REPL tree"] MCP_EP["src/entrypoints/mcp.ts"] SDK_EP["src/entrypoints/sdk/\n28 HOOK_EVENTS, EXIT_REASONS\nZod-generated types from mcp-server"] end subgraph LLM["AREA-2: LLM Orchestration Core"] QE["src/QueryEngine.ts 1297 LOC / 46KB\nClass: one per SDK conversation\nsubmitMessage() async generator\nOwns: messages, file cache, usage\npermission denial tracking"] QUERY["src/query.ts\nCore streaming loop\nTerminal/Continue state machine\nAutocompact + microcompact\nStop hooks"] QDEPS["src/query/deps.ts\nQueryDeps: callModel, microcompact\nautocompact, uuid (injectable)"] QCONF["src/query/config.ts\nQueryConfig: immutable per-query\nRuntime gates snapshot"] QBUDGET["src/query/tokenBudget.ts\nBudgetTracker: 90% threshold\nDiminishing returns detection"] QTRANS["src/query/transitions.ts\nTerminal: completed/error/aborted\nContinue: tool_use/compact/queued"] end subgraph Tools["AREA-3: Tool System"] TOOL_T["src/Tool.ts 794 LOC\nbuildTool factory\nToolUseContext (full exec env)\nToolPermissionContext (DeepImmutable)"] TOOLS["src/tools.ts 396 LOC\ngetAllBaseTools() — 40+ tools\ngetTools(permCtx) — filtered\nassembleToolPool() — sorted+MCP\nPrompt cache stability via sort"] TOOLDIR["src/tools/ 40+ tool dirs\nAgentTool, BashTool, FileRead/Write/Edit\nGlobTool, GrepTool, WebFetch, TodoWrite\nTaskCreate/Get/Update/List, LSPTool\nMCPTool, ListMcpResources, SkillTool"] end subgraph Coord["AREA-4: Multi-Agent Coordination"] CMODE["src/coordinator/coordinatorMode.ts\nCOORDINATOR_MODE feature gate\nINTERNAL_WORKER_TOOLS set\nmatchSessionMode()"] TASKTY["src/Task.ts + tasks/types.ts\n7 types: local_bash, local_agent\nremote_agent, in_process_teammate\nlocal_workflow, monitor_mcp, dream\nTaskStateBase: disk output, status"] TASKDIRS["src/tasks/\nLocalAgentTask, RemoteAgentTask\nInProcessTeammateTask, DreamTask\nLocalShellTask, LocalWorkflowTask\nMonitorMcpTask"] end subgraph Svc["AREA-5: External Services + Bridge"] CLAUDE_SVC["services/api/claude.ts\nqueryModelWithStreaming()\nMulti-provider: Anthropic/Bedrock/GCP\nBeta SDK, prompt cache breakpoints"] COMPACT["services/compact/\nautoCompact, microCompact\nreactiveCompact (REACTIVE_COMPACT)\nsnipCompact (HISTORY_SNIP, SDK-only)"] MCP_SVC["services/mcp/\nMCPConnectionManager\nInProcessTransport\nelicitationHandler\nchannelPermissions"] BRIDGE["src/bridge/ 25 files\nbridgeMain.ts: HTTP poll + JWT\nbridgeApi, bridgeMessaging\nCCR worktree isolation\nfeature(BRIDGE_MODE)"] OTHER_SVC["services/: oauth, lsp, analytics\nGrowthBook, OpenTelemetry, Datadog\nplugins, policyLimits\nremoteManagedSettings, SessionMemory"] end subgraph UI["AREA-6: React/Ink Terminal UI"] INK_FORK["src/ink/ Custom Ink fork\nreconciler, dom, renderer\nTab status, click events\nSearch highlight, FocusManager"] INK_TS["src/ink.ts\nThemeProvider auto-wrap\nPublic API: Box, Text, hooks"] COMPONENTS["src/components/ 144 components\ndesign-system/: ThemeProvider\nThemedBox, ThemedText, color\nPromptInput, MessageResponse\nPermissionRequest, VirtualList"] REPL_SCR["src/screens/REPL.tsx 5006 LOC\nReact Compiler enabled\nIntegration hub: permissions\nMCP elicitation, task nav\nSwarm permission polling"] end subgraph Perms["AREA-7: Permissions & Hooks"] PERM_SYS["src/utils/permissions/ 24 files\n6 modes: default, plan, acceptEdits\nbypassPermissions, dontAsk, auto\nRule parser: Bash(git *) wildcards\nML classifier (TRANSCRIPT_CLASSIFIER)\nbypassKillswitch"] HOOKS["src/hooks/\nuseCanUseTool (central perm check)\nuseSettings, useCommandKeybindings\nfileSuggestions, toolPermission/"] end subgraph Utils["AREA-8: Utilities + Skills + Plugins"] UTILS["src/utils/ 329 modules\nbash, git, auth, settings/\nmodel/, telemetry/, swarm/\nsandbox/, secureStorage/\nstartupProfiler, config, messages"] SKILLS["src/skills/ + src/plugins/\nbundledSkills, loadSkillsDir\nSkillTool, MCP skill builders\nbuiltinPlugins, plugin commands"] end subgraph State["AREA-9: State & Config"] STORE["src/state/\ncreateStore (custom pub-sub)\nAppStateStore (570 LOC)\nSpeculationState (pre-computation)\nselectors, onChangeAppState"] CONFIG["src/schemas/ + src/migrations/\nZod v4 schemas\n10 one-shot startup migrations\n(model name + settings migrations)"] end subgraph External["AREA-10: External Systems"] WEB["web/ — Next.js dashboard\nApp Router, ink-app/\nvercel + helm deployment"] MCP_SRV["mcp-server/ — st

[D] Real-time multi-dimensional LLM output scoring in production, what's actually feasible today?

I'm deep in research on whether a continuous, multi-dimensional scoring engine for LL outputs is production-viable, not as an offline eval pipeline, but as a real-time layer that grades every output before it reaches an end user. Think sub-200ms latency budget across multiple quality dimensions simultaneously. The use case is regulated industries (financial services specifically) where enterprises need provable, auditable evidence that their Al outputs meet quality and compliance thresholds, not just "did it leak Pil" but "is this output actually accurate, is it hallucinating, does it comply with our regulatory obligations." The dimensions I'm exploring: Data exposure - PIl, credentials, sensitive data detection. Feels mostly solved via NER + regex + classification. Low latency, high confidence. Policy violation - rule-engine territory. Define rules, match against them. Tractable. Tone / brand safety - sentiment + classifier approach. Imperfect but workable. Bias detection, some mature-ish approaches, though domain-specific tuning seems necessary. Regulatory compliance, this is where I think domain-narrowing helps. If you're only scoring against ASIC/APRA financial services obligations (not "all regulations everywhere"), you can build a rubric-based eval that's bounded enough to be reliable. Hallucination risk, this is where I'm hitting the wall. The LLM-as-judge approach (RAGAS faithfulness, DeepEval, Chainpoll) seems to be the leading method, but it requires a second model call which destroys the latency budget. Vectara's approach using a fine-tuned cross-encoder is faster but scoped to summarisation consistency. I've looked at self-consistency methods and log-probability approaches but they seem unreliable for production use. Accuracy, arguably the hardest. Without a ground truth source or retrieval context to check against, how do you score "accur V on arbitrary outputs in real time? Is this even a well-defined problem outside of RAG pipelines? My specific questions for people who've built eval pipelines in production: • Has anyone deployed faithfulness/hallucination scoring with hard latency constraints (<200ms)? What architecture did you use distilled judge models, cached evaluations, async scoring with retroactive flagging? • Is the "score everything in real time" framing even the right approach, or do most production systems score asynchronously and flag retroactively? What's the UX tradeoff? • For the accuracy dimension specifically, is there a viable approach outside of RAG contexts where you have retrieved documents to check against? Or should this be reframed entirely (e.g., "groundedness" or "confidence calibration" instead of "accuracy")? • Anyone have experience with multi-dimension scoring where individual classifiers run in parallel to stay within a latency budget? Curious about the infrastructure patterns. I've read through the Datadog LL Observability hallucination detection work (their Chainpoll + multi-stage reasoning approach), Patronus Al's Lynx model, the Edinburgh NLP awesome-hallucination-detection compilation, and Vectara's HHEM work. Happy to go deeper on anything I'm missing. trying to figure out where the technical boundary is between "buildable today" and "active research problem." If anyone has hands on experience here and would be open to a call, I'd happily compensate for your time. submitted by /u/dmc_3 [link] [comments]