Lab 6 - Stress Testing Tutorial: Spring Boot and Express.js

In this tutorial, I’ll guide you through stress testing for the Spring Boot Book Management API and Express.js User Management API projects, building on the previous tutorials (unit, integration, E2E testing, Jira/Xray/Zephyr for Scrum and test management, Confluence for sprint documentation, and GitHub for bug management). Stress testing evaluates how the system performs under extreme conditions, such as high traffic or resource constraints, to identify breaking points and ensure reliability. We’ll use JMeter for stress testing both APIs, integrate the results into our Jira workflow, document findings in Confluence, and manage any identified issues in GitHub.

Overview

Stress testing involves pushing the system beyond its normal operational capacity to determine its breaking point, identify bottlenecks, and ensure it fails gracefully. We’ll:

  1. Use JMeter to simulate high traffic on the Spring Boot and Express.js APIs.
  2. Analyze the results to identify performance issues.
  3. Log issues in Jira, link them to GitHub for resolution, and document the process in Confluence.

We’ll continue with the two projects:

  • Book Management Scrum (BMS): Spring Boot API for managing books, running at http://localhost:8080/api/books.
  • User Management Scrum (UMS): Express.js API for managing users, running at http://localhost:3001/api/users.

Prerequisites

  • Spring Boot and Express.js Projects: Ensure both projects are set up and running as described in the previous tutorials:
    • Spring Boot API: http://localhost:8080/api/books
    • Express.js API: http://localhost:3001/api/users
  • Jira Projects: The Book Management Scrum (BMS) and User Management Scrum (UMS) projects in Jira, with sprints, epics, stories, and test cases managed via Xray/Zephyr.
  • Confluence Spaces: The Book Management Documentation (BMDOC) and User Management Documentation (UMDOC) spaces for documentation.
  • GitHub Repositories: The book-management-api and user-management-api repositories integrated with Jira.
  • JMeter: Download and install Apache JMeter (available at jmeter.apache.org). JMeter is a popular open-source tool for performance and stress testing.

Part 1: Stress Testing the Spring Boot Book Management API

Step 1: Set Up JMeter for Stress Testing

  1. Launch JMeter:

    • Open JMeter by running the jmeter executable (e.g., jmeter.bat on Windows or jmeter.sh on macOS/Linux).
    • A new test plan will open automatically.

  2. Create a Test Plan:

    • Rename the test plan to Book Management Stress Test.
    • Right-click on the test plan > Add > Threads (Users) > Thread Group.
    • Configure the Thread Group:
      • Name: Book API Users
      • Number of Threads (users): 100 (simulates 100 concurrent users)
      • Ramp-Up Period: 10 seconds (users will start over 10 seconds)
      • Loop Count: 10 (each user will repeat the test 10 times)

  3. Add HTTP Request Samplers:

    • Right-click on the Thread Group > Add > Sampler > HTTP Request.
    • Configure the HTTP Request for the GET /api/books endpoint:
      • Name: Get All Books
      • Server Name or IP: localhost
      • Port Number: 8080
      • Path: /api/books
      • Method: GET
    • Add another HTTP Request for the POST /api/books endpoint:
      • Name: Add a New Book
      • Server Name or IP: localhost
      • Port Number: 8080
      • Path: /api/books
      • Method: POST
      • Body Data: {"title": "Stress Test Book ${__threadNum}", "author": "Author ${__threadNum}"}
        • ${__threadNum} ensures each thread creates a unique book.

  4. Add Listeners to View Results:

    • Right-click on the Thread Group > Add > Listener > View Results Tree (to see detailed request/response data).
    • Add another listener: Summary Report (to see aggregated metrics like average response time, throughput, and error rate).

  5. Add Assertions to Validate Responses:

    • Right-click on the Get All Books HTTP Request > Add > Assertions > Response Assertion.
    • Configure the assertion:
      • Apply to: Main sample only
      • Response Field to Test: Response Code
      • Pattern Matching Rules: Equals
      • Patterns to Test: 200
    • Repeat for the Add a New Book request.

Step 2: Run the Stress Test

  1. Start the Spring Boot API:

    • Ensure the Spring Boot application is running at http://localhost:8080/api/books.

  2. Run the Test:

    • In JMeter, click the Start button (green play icon).
    • Monitor the View Results Tree for individual request results and the Summary Report for aggregated metrics.

  3. Analyze the Results:

    • In the Summary Report, look for:
      • Average Response Time: Should be low (e.g., < 200ms for GET requests).
      • Throughput: Requests per second (higher is better).
      • Error %: Percentage of failed requests (should be 0% if all assertions pass).
    • Example results (hypothetical):
      • Get All Books: Average 150ms, Throughput 50/sec, Error 0%
      • Add a New Book: Average 300ms, Throughput 30/sec, Error 5%
    • If errors occur (e.g., 5% of POST requests fail with a 500 error), note the issue for further investigation.

Step 3: Log Issues in Jira and GitHub

  1. Log a Performance Issue in Jira:

    • In the Book Management Scrum (BMS) project, create a new issue:
      • Summary: “POST /api/books fails under high load”
      • Issue Type: Bug
      • Description: “During stress testing with 100 concurrent users, 5% of POST /api/books requests failed with a 500 error. Average response time was 300ms.”
      • Link to Sprint: Sprint 1 - Book CRUD (or the next sprint if Sprint 1 is closed).
      • Assign to a developer (e.g., Fatma).

  2. Create a GitHub Issue:

    • Open the Jira issue (e.g., BMS-124).
    • In the GitHub section, click Create GitHub Issue:
      • Select the book-management-api repository.
      • The issue will be created in GitHub (e.g., book-management-api#3) and linked to BMS-124.

  3. Document Findings in Confluence:

    • In the Book Management Documentation (BMDOC) space, go to the Sprint 1 - Book CRUD page.
    • Add a section for Stress Test Results:
      • Summary: “Stress test with 100 concurrent users showed 5% failure rate on POST /api/books.”
      • Embed the Jira issue (BMS-124) using the Jira Issue/Filter macro.
      • Attach a screenshot of the JMeter Summary Report (export as a CSV and convert to a table in Confluence, or take a screenshot).

Step 4: Fix the Issue

  1. Investigate the Issue:

    • Developer Fatma opens the GitHub issue (book-management-api#3).
    • She identifies that the H2 in-memory database struggles under high load due to concurrent writes, causing the 500 errors.

  2. Implement a Fix:

    • Fatma creates a branch:

      git checkout -b fix-post-high-load

    • She adds a simple retry mechanism in the BookService to handle database contention:

      @Service
public class BookService {
    @Autowired
    private BookRepository bookRepository;

    public Book saveBook(Book book) {
        int retries = 3;
        while (retries > 0) {
            try {
                return bookRepository.save(book);
            } catch (Exception e) {
                retries--;
                if (retries == 0) throw e;
                try {
                    Thread.sleep(100); // Wait before retrying
                } catch (InterruptedException ie) {
                    Thread.currentThread().interrupt();
                }
            }
        }
        return null;
    }
    // Other methods...
}

    • She commits the fix:

      git commit -m "Add retry mechanism for database writes under high load - BMS-124"
git push origin fix-post-high-load

    • She creates a pull request (PR) in GitHub, linking it to the issue (book-management-api#3).

  3. Re-Run the Stress Test:

    • After merging the PR, re-run the JMeter test.
    • Verify that the error rate for Add a New Book drops to 0% and the average response time improves (e.g., to 200ms).

  4. Update Jira and Confluence:

    • The Jira issue (BMS-124) is automatically updated to Done when the PR is merged (via GitHub for Jira integration).
    • In Confluence, update the Stress Test Results section with the new results and a link to the GitHub PR.

Part 2: Stress Testing the Express.js User Management API

Step 1: Set Up JMeter for Stress Testing

  1. Create a New Test Plan:

    • In JMeter, create a new test plan named User Management Stress Test.
    • Add a Thread Group:
      • Name: User API Users
      • Number of Threads: 100
      • Ramp-Up Period: 10 seconds
      • Loop Count: 10

  2. Add HTTP Request Samplers:

    • Add an HTTP Request for GET /api/users:
      • Name: Get All Users
      • Server Name or IP: localhost
      • Port Number: 3001
      • Path: /api/users
      • Method: GET
    • Add an HTTP Request for POST /api/users:
      • Name: Add a New User
      • Server Name or IP: localhost
      • Port Number: 3001
      • Path: /api/users
      • Method: POST
      • Body Data: {"name": "Stress Test User ${__threadNum}", "email": "user${__threadNum}@example.com"}

  3. Add Listeners and Assertions:

    • Add View Results Tree and Summary Report listeners.
    • Add a Response Assertion to each HTTP Request to check for a 200 (GET) or 201 (POST) status code.

Step 2: Run the Stress Test

  1. Start the Express.js API:

    • Ensure the Express.js application is running at http://localhost:3001/api/users.

  2. Run the Test:

    • In JMeter, click Start to run the test.
    • Monitor the results in the View Results Tree and Summary Report.

  3. Analyze the Results:

    • Example results (hypothetical):
      • Get All Users: Average 120ms, Throughput 60/sec, Error 0%
      • Add a New User: Average 400ms, Throughput 25/sec, Error 10%
    • If errors occur (e.g., 10% of POST requests fail due to SQLite database locking), note the issue.

Step 3: Log Issues in Jira and GitHub

  1. Log a Performance Issue in Jira:

    • In the User Management Scrum (UMS) project, create a new issue:
      • Summary: “POST /api/users fails due to database locking under high load”
      • Issue Type: Bug
      • Description: “During stress testing with 100 concurrent users, 10% of POST /api/users requests failed due to SQLite database locking. Average response time was 400ms.”
      • Link to Sprint: Sprint 1 - User CRUD (or the next sprint).
      • Assign to a developer (e.g2. Create a GitHub Issue:
    • Open the Jira issue (e.g., UMS-124).
    • In the GitHub section, click Create GitHub Issue:
      • Select the user-management-api repository.
      • The issue will be created in GitHub (e.g., user-management-api#4) and linked to UMS-124.

  2. Document Findings in Confluence:

    • In the User Management Documentation (UMDOC) space, go to the Sprint 1 - User CRUD page.
    • Add a section for Stress Test Results:
      • Summary: “Stress test with 100 concurrent users showed 10% failure rate on POST /api/users due to SQLite locking.”
      • Embed the Jira issue (UMS-124) using the Jira Issue/Filter macro.
      • Attach a screenshot of the JMeter Summary Report.

Step 4: Fix the Issue

  1. Investigate the Issue:

    • Developer Salah opens the GitHub issue (user-management-api#4).
    • He identifies that SQLite’s default configuration causes locking issues under high load.

  2. Implement a Fix:

    • Salah creates a branch:

      git checkout -b fix-sqlite-locking

    • He modifies database.js to use a write-ahead logging (WAL) mode for SQLite, which improves concurrency:

      const Database = require("better-sqlite3");

const db = new Database(":memory:");
db.pragma("journal_mode = WAL"); // Enable WAL mode for better concurrency

db.exec(`
    CREATE TABLE users (
        id INTEGER PRIMARY KEY AUTOINCREMENT,
        name TEXT NOT NULL,
        email TEXT NOT NULL
    )
`);

module.exports = db;

    • He commits the fix:

      git commit -m "Enable WAL mode for SQLite to handle high load - UMS-124"
git push origin fix-sqlite-locking

    • He creates a pull request in GitHub, linking it to the issue (user-management-api#4).

  3. Re-Run the Stress Test:

    • After merging the PR, re-run the JMeter test.
    • Verify that the error rate for Add a New User drops to 0% and the average response time improves (e.g., to 150ms).

  4. Update Jira and Confluence:

    • The Jira issue (UMS-124) is updated to Done when the PR is merged.
    • In Confluence, update the Stress Test Results section with the new results and a link to the GitHub PR.

Hands-On Labs

Lab 1: Increase Load and Analyze Breaking Point

  • Task: In the Book Management Stress Test, increase the number of threads to 500 and the loop count to 20. Identify the breaking point of the Spring Boot API and log a new issue in Jira.
  • Steps:
    1. Update the Thread Group in JMeter: 500 threads, 20 loops.
    2. Run the test and analyze the Summary Report (e.g., error rate increases to 20%, response time spikes to 2 seconds).
    3. Log a bug in Jira (BMS-125): “API crashes with 500 concurrent users.”
    4. Create a GitHub issue and document the findings in Confluence.

Lab 2: Stress Test a New Endpoint

  • Task: Add a new endpoint to the Express.js API (e.g., PUT /api/users/:id to update a user) and stress test it with JMeter.
  • Steps:
    1. Add the endpoint in app.js: 
      app.put("/api/users/:id", (req, res) => {
  const { name, email } = req.body;
  const update = db.prepare(
    "UPDATE users SET name = ?, email = ? WHERE id = ?"
  );
  const result = update.run(name, email, req.params.id);
  if (result.changes === 0) {
    return res.status(404).json({ message: "User not found" });
  }
  const updatedUser = db
    .prepare("SELECT * FROM users WHERE id = ?")
    .get(req.params.id);
  res.json(updatedUser);
});
    2. In JMeter, add an HTTP Request to the User Management Stress Test:
      • Path: /api/users/1
      • Method: PUT
      • Body Data: {"name": "Updated User ${__threadNum}", "email": "updated${__threadNum}@example.com"}
    3. Run the test, analyze the results, and log any issues in Jira.

Lab 3: Document Stress Test Results in Confluence

  • Task: Create a dedicated Stress Testing page in the Book Management Documentation space to summarize all stress test results and link to related Jira issues.
  • Steps:
    1. Create a new page under Sprint Documentation called Stress Testing.
    2. Add sections for each stress test run (e.g., initial test, 500-user test).
    3. Embed Jira issues (BMS-124, BMS-125) and attach JMeter result screenshots.

Key Concepts and Best Practices

  1. Stress Testing with JMeter:
    • Use Thread Groups to simulate concurrent users and ramp-up periods to mimic real-world traffic patterns.
    • Add assertions to validate responses and ensure the system fails gracefully.
    • Analyze metrics like response time, throughput, and error rate to identify bottlenecks.
  2. Integration with Jira and GitHub:
    • Log performance issues as bugs in Jira and link them to GitHub for resolution, ensuring traceability.
    • Use GitHub commits and PRs to automatically update Jira issue statuses, streamlining the workflow.
  3. Documentation in Confluence:
    • Document stress test results in Confluence to provide a historical record and share findings with stakeholders.
    • Embed Jira issues and test reports to maintain a single source of truth.
  4. Iterative Improvement:
    • Use stress test results to iteratively improve the system, addressing bottlenecks and ensuring scalability.

Additional Notes

  • Scaling Stress Tests: For production systems, run stress tests in a staging environment that mirrors production to get accurate results. Use tools like JMeter’s distributed testing for larger loads.
  • Monitoring: Integrate monitoring tools (e.g., Prometheus, Grafana) to observe system metrics (CPU, memory, database connections) during stress tests.
  • Automation: Automate stress tests in a CI/CD pipeline (e.g., using Jenkins) and integrate results with Xray/Zephyr for test execution reporting.

This tutorial provides a comprehensive guide to stress testing the Spring Boot and Express.js APIs, integrating the results into your existing Jira, Confluence, and GitHub workflow. By completing the labs, you’ll gain hands-on experience in identifying and resolving performance issues, ensuring your applications can handle high loads.

By Wahid Hamdi