How to Fix Next.js Broken Pipe on Ubuntu 20.04

Troubleshooting “Next.js Broken Pipe” on Ubuntu 20.04

As a Senior DevOps Engineer, encountering a “Broken Pipe” error with Next.js applications, especially during next build or next dev processes on Ubuntu 20.04, is a common signal of resource contention. This guide will walk you through diagnosing and resolving this issue efficiently, ensuring the stability of your development and deployment workflows.

1. The Root Cause: Understanding File Descriptor Limits

The “Broken Pipe” error, in the context of a Next.js application on Linux, is most frequently a symptom of an underlying issue: exceeding the maximum number of open file descriptors allowed for a process.

Why this happens on Ubuntu 20.04 (and similar Linux distributions): Linux systems, by default, impose limits on the resources a process can consume. One critical limit is the nofile (number of open files) ulimit, which dictates how many file descriptors a process can have open simultaneously. A typical default nofile limit on Ubuntu 20.04 for non-root users is 1024.

Next.js, particularly during complex operations like:

  • next build: Compiling numerous pages, processing static assets, optimizing images, and resolving a large dependency tree (e.g., node_modules).
  • next dev: Hot Module Replacement (HMR) and file watching mechanisms.

Each file opened, watched, or processed consumes a file descriptor. With a modern Next.js application, especially one with many pages, components, or a substantial node_modules directory, it’s very easy for the build process (which heavily leverages tools like Webpack, Babel, and PostCSS) to exceed the default 1024 file descriptor limit. When this limit is hit, subsequent attempts to open files or establish inter-process communication (which also uses file descriptors) can fail, manifesting as “EMFILE: too many open files” or the more generic “Broken Pipe” error.

2. Quick Fix (CLI): Temporary Upping the Limits

For immediate relief and to confirm that file descriptor limits are indeed the culprit, you can temporarily increase the nofile limit for your current shell session.

  1. Open your terminal.
  2. Increase the nofile limit: We recommend setting it to a significantly higher value, such as 16384 or 65536, especially for build processes. 
    ulimit -n 16384
    • Note: This change is only valid for the current shell session. If you open a new terminal or restart your machine, the limit will revert.
  3. Run your Next.js command: Now, attempt to run your Next.js build or development server again in the same terminal where you set the ulimit. 
    # For building your application
npm run build
# or
yarn build

# For running the development server
npm run dev
# or
yarn dev

If your Next.js process now completes successfully, you’ve confirmed that the file descriptor limit was the issue. The next step is to make this change persistent.

3. Configuration Check: Making Limits Permanent

To prevent this issue from recurring, you need to configure your system to allocate higher file descriptor limits permanently. The method depends on how your Next.js application is being run.

Method 1: System-Wide for User Sessions (/etc/security/limits.conf)

This is the most common approach for development machines or scenarios where the application is run directly by a user.

  1. Edit limits.conf: Open the /etc/security/limits.conf file with sudo privileges. 
    sudo nano /etc/security/limits.conf
# or
sudo vim /etc/security/limits.conf
  2. Add or modify lines: Add the following lines at the end of the file. The * wildcard applies these limits to all users. If you want to limit it to a specific user (e.g., your_username), replace * with that username. 
    # Increase file descriptor limit for all users
*   soft    nofile    16384
*   hard    nofile    16384
    • soft: The current active limit. Processes can increase their soft limit up to the hard limit.
    • hard: The maximum limit a process can set for its soft limit.
    • A value of 16384 is generally sufficient, but you can increase it further (e.g., 65536) if issues persist or for very large projects.
  3. Save and exit.
  4. Verify PAM configuration (usually default): Ensure that Pluggable Authentication Modules (PAM) are configured to use pam_limits.so. This is typically enabled by default on Ubuntu. You can check the following files for a line like session required pam_limits.so:
    • /etc/pam.d/common-session
    • /etc/pam.d/common-session-noninteractive
  5. Reboot or Re-log: For these changes to take effect, you must either reboot your system or log out and log back in to your user session.

Method 2: For Systemd Services (.service files)

If your Next.js application is deployed as a systemd service (common for production environments), you need to specify the limit within its service unit file.

  1. Locate your service file: This is typically found in /etc/systemd/system/your-nextjs-app.service. 
    sudo nano /etc/systemd/system/your-nextjs-app.service
  2. Add LimitNOFILE: Within the [Service] section, add the LimitNOFILE directive. 
    [Service]
# ... other directives ...
LimitNOFILE=16384
# ... other directives ...
    Again, adjust 16384 as needed.
  3. Save and exit.
  4. Reload Systemd and Restart Service: Apply the changes and restart your Next.js service. 
    sudo systemctl daemon-reload
sudo systemctl restart your-nextjs-app.service

4. Verification: Confirming the Fix

After applying the permanent changes, it’s crucial to verify that the new limits are in effect and that your Next.js application now functions correctly.

  1. Check ulimit for your session (after re-login/reboot): Open a new terminal session (or log in again if you edited limits.conf).

    ulimit -n

    The output should reflect the new, higher value (e.g., 16384).

  2. Check ulimit for the running Next.js process (if applicable): If your Next.js application is already running as a service or in the background, you can inspect its specific limits.

    • First, find the Process ID (PID) of your Next.js application: 
      ps aux | grep "node" | grep "next"
# Look for the main node process running your Next.js app, e.g., node server.js
    • Once you have the PID (let’s say it’s 12345), check its limits: 
      cat /proc/12345/limits
      Look for the Max open files line; it should show your configured soft and hard limits.

  3. Re-run Next.js operations: Execute npm run build or npm run dev again, ensuring no “Broken Pipe” or “Too many open files” errors occur.

By following these steps, you should successfully resolve “Next.js Broken Pipe” errors on Ubuntu 20.04, ensuring a smooth development and deployment experience for your Next.js applications. This proactive adjustment of file descriptor limits is a standard practice for resource-intensive Node.js applications in production and development environments alike.

⚡ Related Troubleshooting Guides