Add S3 Hook to Enable Zerocode-TDD Steps for Upload, List, and Download Scenarios #742

Background

Zerocode-TDD enables highly modular test scenario composition. Adding S3 step hooks will allow users to perform file operations within their test workflows straight from scenarios.

Problem / Requirement

Enable S3 operations via Zerocode-TDD steps. The DSL/workflow must support:

• S3 upload
• S3 list
• S3 download

Proposed DSL Workflow & Structure

Suggest the workflow for an easy and clear S3 integration:

Connection & Credentials

• Should be defined in a config file or as part of environment/step, e.g. s3_host.properties. File should contain:

  s3.endpoint=https://s3.amazonaws.com <--- Keep it optional in the S3 Client builder, use this if present
  s3.accessKey=<your-access-key>
  s3.secretKey=<your-secret-key>
  s3.region=<region>
  s3.token=<temporary token if required> <--- This is optional only. Handle both 1)with this and 2)without this token.

• "s3.endpoint": Cater for both situations 1)with s3.endpoint 2)without s3.endpoint. Just build the S3 client accordingly.

• As usual, can use Zerocode's @TargetEnv or @HostProperties annotation or scenario properties referencing this config.

S3 Client implementation:

1st check if the credentials present in the env variables, otherwise check the ".properties" file:
example:

        // ---------- Environment variables ----------
        String envAccessKey = System.getenv("AWS_ACCESS_KEY_ID");
        String envSecretKey = System.getenv("AWS_SECRET_ACCESS_KEY");
        String envSessionToken = System.getenv("AWS_SESSION_TOKEN");

eg:
if "envSessionToken" empty or null, then connect using envAccessKey and envSecretKey.

Step Structure Examples

• Upload File:

  {
    "name": "upload_to_S3",
    "operation": "S3.UPLOAD",
    "request": {
      "bucket": "test-bucket", <---Also allow "S3://test-bucket"
      "key": "folder/file.txt",
      "filePath": "data/input.txt"
    },
    "verify": {
      "status": 200 //<---- or "OK" or "SUCCESS" whatever is appropriate for S3 response
    }
  }

• List Files:

  {
    "name": "list_files_in_S3",
    "operation": "S3.LIST",
    "request": {
      "bucket": "test-bucket",
      "folder": "folder/"
    },
    "verify": {
      "files.SIZE": 2, 
      "files[0].key": "folder/file.txt"
    }
  }

• Download File:

  {
    "name": "download_from_S3",
    "operation": "S3.DOWNLOAD",
    "request": {
      "bucket": "test-bucket",
      "key": "folder/file.txt",
      "saveAs": "data/downloaded.txt" <--- If it doesn’t start with /, it goes to "target/data.."; if it starts with /, it’s absolute.
    },
    "verify": {
      "downloaded": true
    }
  }

Workflow Recommendation

• Credentials/configs: As usual, keep it outside the step, referenced in scenario or step file.
• Each step represents one S3 operation (similar to HTTP or Kafka steps).
• Use step file for batch uploads if needed. <------- Create a new Issue for this.
• Each step can use verify/assertion block to check results.

Acceptance Criteria

1. Zerocode-TDD DSL allows S3 upload, list, and download operations as scenario steps.
2. S3 credentials/config can be referenced as scenario-level or external file (s3_host.properties).
3. Steps include clean request, operation, verify blocks as in example above.
4. Upload step takes a filePath and uploads to bucket/key.
5. List step returns list of keys/objects, filters supported.
6. Download step downloads key and saves as local file.
7. Errors (auth, bucket missing) are reported in verify/assertion block.
8. Full workflow can be run headless and reports results per step.

Additional Notes

• File and folder support for list/upload/download
• Examples, docs, and sample scenario files will be provided with the PR.
• Open for feedback on the DSL patterns above.

Phase 2

Raise a new ticket(GitHub Issue) for this:
if random(dynamic) placeholders were used in the input file(for upload), example {{RANDOM_VALUE1}} or {{EMPLOYEE_ID1}}, then resolve them before uploading and include them in the response as well.
i.e. Implement the logic to generate random values for these dynamic placeholders and save this context so that once the values are generated(eg. in CSV or any input text file), they can be reused for assertions.

    "verify": {
      "status": 200, //<---- or "OK" or "SUCCESS"
      "tokens": {
          "RANDOM_VALUE1": 30000055,
          "EMPLOYEE_ID2": 30000055
      }
}

See README for current DSL, and extend similar approach for S3. The test config pattern is shown in HelloWorldFileUploadTest, follow the simillar pattern for S3.

Documentation:

Update in this doc repo here.