Migrating Distributed Press from Kubo to Helia: A Developer’s Guide

Backed by the IPFS Utility grant, we ran an experiment: moving from Kubo to Helia to see if we could level up our API’s performance and make things smoother for developers. This report walks through the process, highlights key takeaways, and shares practical advice for anyone thinking about building on Helia.

Introduction

Distributed Press, a platform for decentralized publishing, embarked on a migration experiment from Kubo to Helia to enhance its API’s performance and developer experience, as part of our IPFS Utility grant-funded work. We chose Helia for its lightweight, JavaScript-native architecture, which promised faster initialization, reduced resource usage, and greater modularity compared to Kubo’s monolithic design. This migration, detailed in PR #101, aimed to streamline content publishing and improve maintainability. However, the process revealed several configuration challenges, from DHT advertisement failures to connectivity issues. This report shares our journey, performance comparisons, and practical guidance to help developers build robust Helia-based applications, avoiding the pitfalls we encountered.

Performance Comparison: Helia vs. Kubo

We measured initialization and end-to-end (E2E) publish operation times to compare Helia and Kubo, focusing on their efficiency in a server environment:

Metric

Kubo

Helia

Notes

Initialization

10-13s

150–550ms

Helia’s lightweight design reduces startup time compared to Kubo’s monolithic setup.

E2E Publish

900ms–2s

160–800ms

Helia’s streamlined DHT interactions speed up content upload and advertisement.

Helia’s faster initialization stems from its modular architecture, which avoids loading unnecessary components like Kubo’s built-in gateway. The E2E publish performance benefits from optimized Libp2p configurations, though proper tuning was critical to achieving these results.

Helia’s Customizability with Libp2p

Helia’s integration with Libp2p offers developers full control over the networking stack by allowing direct injection of a Libp2p instance, unlike Kubo, which embeds and configures go-libp2p internally through a limited set of exposed JSON settings. As a binary, Kubo abstracts much of the networking configuration behind its own API, gateway, and swarm settings, which can be limiting when trying to customize or optimize behavior. In contrast, Helia, built on Libp2p’s modular stack, allows developers to select transports (e.g., TCP, WebSockets, WebRTC), encryption (e.g., Noise), and peer discovery mechanisms (e.g., bootstrap, mDNS). This modularity enabled us to tailor our node for server-side publishing, enabling WebRTC for NAT traversal and configuring kadDHT for server-mode operation. However, this flexibility requires careful configuration to avoid issues like private IP advertisement or connectivity failures, as we learned during our migration.

Development Guidance

Our migration from Kubo to Helia revealed several pain points that developers can avoid by following these recommendations, drawn from our challenges documented in PR #101.

Start with Libp2p Defaults

Helia’s default Libp2p configurations provide a solid foundation for peer-to-peer networking, reducing setup complexity. Key resources include:

  • Node.js Defaults: Configures TCP, WebSockets, Noise encryption, and Yamux/Mplex stream muxers for server environments.

  • Browser Defaults: Optimizes for WebRTC and WebSockets, ideal for client-side applications.

Key Libp2p/Helia Configurations and Their Purpose:

Component

Purpose

tcp

Enables TCP transport for reliable peer connections (TCP Docs).

webSockets

Supports WebSocket connections for browser and server compatibility (WebSockets Docs).

webRTC, webRTCDirect

Facilitates NAT traversal for peers behind firewalls (WebRTC Docs).

circuitRelayTransport

Enables relaying through other nodes for connectivity (Circuit Relay Docs).

noise

Provides encryption for secure peer communication (Noise Docs).

yamux

Manages multiplexing for multiple streams over a single connection (Yamux Docs).

kadDHT

Implements Kademlia DHT for peer and content discovery (KadDHT Docs).

bootstrap

Connects to predefined peers for initial network discovery (Bootstrap Docs).

autoNAT

Detects if the node is publicly dialable to inform other modules like DHT (AutoNAT Docs).

uPnPNAT

Uses UPnP for automatic port mapping (UPnP Docs).

identify, identifyPush

Shares peer identity and updates (Identify Docs).

ping

Tests peer connectivity (Ping Docs).

Recommendation: Start with these defaults and customize only as needed; import { libp2pDefaults } from 'helia'

Use // @ts-check for JavaScript

For developers using JavaScript, adding // @ts-check at the top of your Helia configuration file enables type checking without requiring a full TypeScript setup. This caught several errors in our migration, such as incorrect kadDHT options, saving significant debugging time.

Avoid Private IP Advertisement

One of the challenges was the node advertising private IPs (e.g., 127.0.0.1, 10.x.x.x) in the DHT, leading to "gater disallows connection" errors. This occurred because we were not announcing our public IPs in the announce list, causing the node to default to private addresses shared during the identify process. As clarified in How does js-libp2p deal with private networks and IPs?, all listening addresses are sent during identify, and kadDHT can filter private addresses with removePrivateAddressesMapper if needed (e.g., as implemented in the Amino DHT). However, the root issue was the lack of public IP announcement. Take a look at js-libp2p-amino-dht-bootstrapper for how we configure a public js-libp2p node. Additionally, filtering private IPs isn’t always necessary if public peers can connect via dial-back or circuit relay reservations, though expired reservations (e.g., during IPFS checks) could still cause issues. To address this:

  • Use removePrivateAddressesMapper in kadDHT to filter private IPs.

Example from our config:

addresses: {
  announce: [`/ip4/${publicIP}/tcp/${tcpPort}`, `/ip4/${publicIP}/tcp/${wsPort}/ws`],
}

Kubo vs. Helia Binding

Kubo’s default configuration binds its API and gateway to 127.0.0.1 for security, while its swarm uses 0.0.0.0 for peer connections.
API: /ip4/127.0.0.1/tcp/${apiPort}

Helia, lacking a built-in API or gateway, requires explicit binding to 0.0.0.0 for external access. We updated our Ansible configuration (distributed_press_host: "0.0.0.0") to allow external connections, resolving issues like "dial backoff" errors seen in IPFS checks.

Recursive Directory Upload & Pinning

For projects that need to add and persist entire directory trees, Helia’s globSource utility makes it easy to recursively upload folders and then pin the resulting root CID (and all child CIDs) in one step.

import { globSource } from 'helia/unixfs'
import { createHelia } from 'helia'

async function addDirectory(dirPath) {
  // Initialize your Helia node (once)
  const helia = await createHelia()
  // Recursively add all files under dirPath via glob pattern
  const rootCid = await helia.addAll(globSource(dirPath, '**/*'))
    .then(results => {
      // The last result corresponds to the directory itself
      const last = Array.from(results).pop()
      return last.cid
    })
  
  // Pin the directory CID (and all children) in one go
  await helia.pins.add(rootCid, { recursive: true })
  console.log(`Directory ${dirPath} added and pinned at CID: ${rootCid}`)
  return rootCid
}

Additional Tips

  1. Connectivity Testing with IPFS Check: IPFS Check proved to be an invaluable resource for testing node connectivity and content advertisement. This tool provided real-time insights into peer connections and DHT propagation, significantly streamlining our debugging process and ensuring our Helia node was correctly configured for external access.

  2. Logging: Extensive logging (e.g., ctx?.logger.info) helped diagnose issues like failed DHT provides or IPNS resolutions.

  3. Firewall Rules: Ensure all the ports are open in your firewall configuration, as we updated in our Ansible.

  4. Test Timeout: Increase test framework timeouts to accommodate Helia’s network operations, especially in CI environments.

Conclusion

Migrating to Helia improved our API’s performance and developer experience, but required careful configuration to avoid pitfalls like private IP advertisement and timeout issues. By starting with Libp2p defaults, enabling type checking, and tuning DHT and connectivity settings, developers can build robust Helia-based applications. Our experience, documented in PR #101, provides a roadmap for others to follow, ensuring efficient and reliable IPFS integration.

Post by