Melts | Repository Download CLI Tool | Fluid Attacks Help

Introduction

Melts allows downloading client source code repositories. It is mainly used by Fluid Attacks' hackers.

Requirements

Make sure you have Nix installed.

SSH key configuration

Before running Melts, you need to configure SSH access to the GitLab repository.
Get the SSH Key: Copy the SSH key contents from this Google Drive document.

Create an SSH key file

Option 1: If you don't have any SSH keys yet in ~/.ssh/
  1. Create the file ~/.ssh/id_ed25519 and paste the key into it using any editor.
  2. Set correct permissions:
    chmod 600 ~/.ssh/id_ed25519
Option 2: If you already have SSH keys
If you already have SSH keys in ~/.ssh/, create a separate key file for Universe:
  1. Create the file ~/.ssh/universe_external and paste the key into it using any editor
  2. Set correct permissions:
chmod 600 ~/.ssh/universe_external

API Token

Make sure you have an API token from the platform and that you set its value in an environment variable named INTEGRATES_API_TOKEN.

Installation and usage

Method 1

Using default SSH key (id_ed25519):

  1. Check if the SSH key is correct. 
    2. $ ssh -T git@gitlab.com
    Welcome to GitLab, @dsalazaratfluid!
  1. Execute and install melts
    2. nix run "git+ssh://git@gitlab.com/fluidattacks/universe?dir=melts



  1. Repositories in a given group can be downloaded with
    2. nix run "git+ssh://git@gitlab.com/fluidattacks/universe?dir=melts" -- pull-repos --group {group_name}
  2. To download a specific repository, run:
      nix run "git+ssh://git@gitlab.com/fluidattacks/universe?dir=melts -- pull-repos --group {group_name} --root {nickname}




Using custom SSH configuration (universe_external)

  1. Create or edit ~/.ssh/config and add

    2. Host gitlab-universe
         HostName gitlab.com
         User git
         IdentityFile ~/.ssh/universe_external
         IdentitiesOnly yes
    chmod 600 ~/.ssh/config
  1. Configure Git to automatically replace the official URL of the universe repository with the alias gitlab-universe. This ensures that Nix uses the correct SSH key (universe_external) defined in your ~/.ssh/config file:
    2. git config --global url."git@gitlab-universe:fluidattacks/universe".insteadOf "git@gitlab.com:fluidattacks/universe"

  2. Check if the SSH key is correct. 
    3. $ ssh -T git@gitlab-universe
    Welcome to GitLab, @dsalazaratfluid!
  1. Execute and install melts
    2. nix run "git+ssh://git@gitlab-universe/fluidattacks/universe?dir=melts
    1. Repositories in a given group can be downloaded with
      2. nix run "git+ssh://git@gitlab-universe/fluidattacks/universe?dir=melts" -- pull-repos --group {group_name}
    2. To download a specific repository, run:
      nix run "git+ssh://git@gitlab-universe/fluidattacks/universe?dir=melts -- pull-repos --group {group_name} --root {nickname}

Method 2

Using custom SSH configuration (universe_external)

  1. After setting the key, test the SSH connection:
$ ssh -i ~/.ssh/universe_external -o IdentitiesOnly=yes -T git@gitlab.com
Welcome to GitLab, @dsalazaratfluid!
  1. Execute and install Melts
  1. You can configure an alias to run melts:
    > alias melts='GIT_SSH_COMMAND="ssh -i ~/.ssh/universe_external -o IdentitiesOnly=yes" nix run "git+ssh://git@gitlab.com/fluidattacks/universe?dir=melts"'
    >     melts
    > melts pull-repos --group {group_name}
    > melts     pull-repos --group {group_name} --root {nickname}
Notes
Alias should be configured in each new console.

Troubleshooting

In case you encounter any errors while using Melts, there are a couple of things you can try to fix them:
  1. The first thing you should do is follow the installation instructions again.
  2. The next thing you can check is whether your INTEGRATES_API_TOKEN has not expired, for this you only need to repeat the steps shown here for updating your API token, and be aware of when it will expire next.
  3. Another thing that may be causing issues is a conflict in your environment variables that are taken when you log into AWS, so you can try deleting this information and logging in again. In order to do this, use the command rm -rf ~/.aws/credentials before logging in; if that doesn’t work, then use rm -rf ~/.okta* as well. After doing this and logging in with the appropriate credentials and choosing the correct role, if applicable, you should have solved any problems regarding permissions.
  4. If none of these work, get in contact with the Fluid Attacks team by sending an email to help@fluidattacks.com to assist you with any problems.

Architecture

Fluid Attacks Melts architecture
  1. Melts is a CLI written in Python.
  2. It is distributed to the End Users using Nix.
  3. It downloads repositories from clients.
  4. It communicates with the Integrates API to pull data required for downloading such repositories.
  5. It reports execution bugs to Bugsnag.

Contributing

Please read the contributing page first.

Development environment

Configure your Development environment.

When prompted for an AWS role, choose dev, and when prompted for a Development Environment, pick melts.

Local environment

Run this command within the universe/melts directory:

nix run .

This will build and run the Melts CLI application, including the changes you’ve made to the source code.

Idea
Tip
Have an idea to simplify our architecture or noticed docs that could use some love? Don't hesitate to open an issue or submit improvements.