Need

Verification of file integrity and compatibility

Context

• Usage of Go 1.16 for developing high-performance and concurrent applications
• Usage of gin-gonic/gin for building web applications in Go

Description

Non compliant code

        package main

import (
    "github.com/gin-gonic/gin"
    "io/ioutil"
    "net/http"
    "os"
    "path/filepath"
)

func main() {
    r := gin.Default()

    r.POST("/upload", func(c *gin.Context) {
        file, _ := c.FormFile("file")
        c.SaveUploadedFile(file, filepath.Join(".", file.Filename))
        c.String(http.StatusOK, "File uploaded successfully")
    })

    r.GET("/download/:filename", func(c *gin.Context) {
        filename := c.Param("filename")
        content, _ := ioutil.ReadFile(filename)
        c.Data(http.StatusOK, "application/octet-stream", content)
    })

    r.Run(":8080")
}
        
        

In the above code, the application accepts a file upload via the POST request at the endpoint "/upload". The uploaded file is saved in the current directory with its original filename. The application also allows downloading of the file through the GET request at the endpoint "/download/:filename".

The vulnerability lies in the fact that the application does not verify the content of the uploaded file against its extension. This means that an attacker could upload a file with a malicious payload but with a benign extension. For example, an attacker could upload a file named "image.jpg" but the actual content could be a script or executable.

When the application serves this file to the user for download, the user's system could recognize the file by its content rather than its extension and execute the malicious payload. This is especially dangerous if the application is running with elevated privileges as it could lead to a full system compromise.

To mitigate this vulnerability, the application should implement a mechanism to verify that the content of the uploaded file matches its extension. For example, if a user uploads a .jpg file, the application should check that the file content is indeed a JPEG image.

Steps

• Validate file extensions: Implement a validation mechanism to ensure that the file extension matches the actual content of the file. This can be done by checking the file signature or using a library that can verify the file type.
• Restrict file types: Limit the types of files that can be uploaded to the server. Define a whitelist of allowed file extensions and reject any files that do not match the allowed types.
• Sanitize file names: Ensure that the file names are sanitized to prevent any malicious characters or path traversal attacks. Remove any special characters or spaces from the file names.
• Store files in a secure location: Store the uploaded files in a separate directory outside of the application's root directory. This prevents the execution of any potentially malicious files.
• Implement access controls: Restrict access to the uploaded files by implementing proper access controls. Only allow authorized users to download the files.
• Implement file validation on download: Before serving the file for download, validate the file again to ensure its integrity and prevent serving unverifiable files.
• Regularly update file verification mechanisms: Keep the file verification mechanisms up to date by regularly updating the libraries or tools used for file validation.

Compliant code

        package main

import (
    "github.com/gin-gonic/gin"
    "github.com/h2non/filetype"
    "io/ioutil"
    "net/http"
    "os"
    "path/filepath"
    "strings"
)

func main() {
    r := gin.Default()

    r.POST("/upload", func(c *gin.Context) {
        file, _ := c.FormFile("file")
        filename := filepath.Base(file.Filename)
        filename = strings.ReplaceAll(filename, " ", "_")
        filename = strings.ReplaceAll(filename, "..", "")
        filename = strings.ReplaceAll(filename, "/", "")
        filename = strings.ReplaceAll(filename, "\\\\", "")

        allowedExtensions := []string{".jpg", ".png", ".gif", ".pdf"}

        if !contains(allowedExtensions, filepath.Ext(filename)) {
            c.String(http.StatusBadRequest, "File type not allowed")
            return
        }

        path := filepath.Join("secure_directory", filename)
        c.SaveUploadedFile(file, path)

        kind, _ := filetype.MatchFile(path)
        if kind == filetype.Unknown {
            os.Remove(path)
            c.String(http.StatusBadRequest, "Cannot verify file type")
            return
        }

        c.String(http.StatusOK, "File uploaded successfully")
    })

    r.GET("/download/:filename", func(c *gin.Context) {
        filename := c.Param("filename")
        path := filepath.Join("secure_directory", filename)

        kind, _ := filetype.MatchFile(path)
        if kind == filetype.Unknown {
            c.String(http.StatusBadRequest, "Cannot verify file type")
            return
        }

        content, _ := ioutil.ReadFile(path)
        c.Data(http.StatusOK, "application/octet-stream", content)
    })

    r.Run(":8080")
}

func contains(slice []string, item string) bool {
    for _, a := range slice {
        if a == item {
            return true
        }
    }
    return false
}
        
        

The updated code includes several security measures to prevent the upload and download of unverifiable files.

1. File Extension Validation: The code checks the file extension against a whitelist of allowed extensions. If the file extension is not in the whitelist, the upload is rejected.

2. File Name Sanitization: The code removes any special characters, spaces, and path traversal sequences from the file name. This prevents potential path traversal attacks.

3. Secure File Storage: The uploaded files are stored in a separate directory named "secure_directory", which is outside of the application's root directory. This prevents the execution of potentially malicious files.

4. File Type Verification: The code uses the filetype library to verify the file type both during upload and before download. If the file type cannot be verified, the upload or download is rejected.

5. Access Controls: The code does not include specific access controls, as these would depend on the specifics of your application. However, you should implement appropriate access controls to ensure that only authorized users can upload and download files.

6. Regular Updates: The filetype library used for file type verification is regularly updated, helping to ensure that the file verification mechanisms remain up-to-date.

References

• 117. Unverifiable files