Claude 3.5 Sonnet | Stack | Fluid Attacks Help

Claude 3.5 Sonnet

Rationale

Claude powers the AI-assisted remediation guides and fixes.

The main reasons why we chose it over other alternatives are the following:
  1. It has ample documentation and prompt engineering guides and examples to improve results.
  2. It ranks within the top 10 models of the BigCodeBench Hard set's leaderboard sorted by the Instruct score, meaning that it performs very well in complex code generation based on NL-oriented instructions.
  3. It can be consumed through Amazon Bedrock which allows us to
    1. use a unified API for many different, top-scoring models of the aforementioned benchmark, and swap them in or out as needed;
    2. add Guardrails to deal with sensitive information;
    3. serve RAG-enhanced answers.

Alternatives

The following alternatives were considered but not chosen for the following reasons:
  1. GPT-4o: Concerns over responsible use and possible violation of copyright laws.
  2. Gemini Pro: Cannot be consumed through the Bedrock API, though it remains a strong contender to consider in the future.
  3. Athene-V2: Promising fine-tuned model based on Qwen but has little documentation and support.
  4. DeepSeek: Has a decent documentation though it cannot be consumed through the Bedrock API. The base Claude model, at the time of writing, also performs better in code generation.

Usage

We use Claude 3.5 Sonnet to power custom and suggested vulnerability fixes, which users can request through the VS Code extension or within our platform.