Project Article - Tech Blog

Backstory

+ I was cleaning out a closet and I discovered an old Steam Controller I bought about a decade ago. To my surprise, it still turned on. Way to go, Duracell. I remembered it being a pretty niche thing that had very little adoption and was quickly deprecated by Valve. Boy was I wrong. I turned it on and immediately remembered how amazing the haptic feedback was on the touchpads. How could I forget the grip inputs? Even a gyroscope! The design and hardware were top notch, but adoption was the problem. +

+ +

+ The haptic feedback touchpads made Games like Civ and XCOM playable on the couch. But I had my eyes on something much bigger target that could use a controller with fourteen buttons, six directional axes, two analog triggers, and a gryoscope: Flight Simulator. With traditional controllers, you had to have a plethora of different button configurations to make it work. Joysticks make it hard to sense and control your aircraft. Touchpads allow for refined movement with feedback to help you ramp up and down with ease. +

+ +

Flight Simulator

+ Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vestibulum eget tortor massa. Sed sed nulla vitae enim accumsan pulvinar fermentum nec nunc. Phasellus scelerisque sem vel massa tempus, id lobortis arcu mattis. Donec sed lectus cursus libero auctor facilisis. Donec mattis luctus congue. Maecenas et nunc odio. Sed eu diam lectus. Phasellus sit amet mi vitae lacus rhoncus elementum. Nullam dictum justo sed dignissim placerat. Pellentesque porta velit vel luctus efficitur. +

+ +

WebSocket Server

+ +

Handles client connections and disconnections gracefully
Broadcasts operations to all connected clients in a room
Implements heartbeat mechanisms to detect stale connections
Scales horizontally using Redis pub/sub for multi-server deployments

+ +

Physical Applications

Parsing the Byte Stream

+ If I want to make a driver for this controller, I need to see what the output is. Controllers are just like a keyboard: character devices. When you type an "a" character into your keyboard, the keyboard sends data for an "a" key to the computer. When you press an "a" button on your controller, it does just about the same thing. It sends data, which in this case is 41 in hex or 01000010 in binary, to the computer and it is handled by the OS, which monitors for any special inputs, before sending it to the application you're using. This happens so quickly, we don't even think about it. It's a little more complicated than that, but I'll explain later. What I need to do is reverse engineer what data the inputs on my controller output and write a driver to turn those inputs into actions. +

+ Knowing my controller is a character device, I can "read" it just like a file, but I have to jump through a couple hoops first. I don't know what that actual file is yet. I'm going to have to check around. When you plug a device into a computer running an OS, it has to decide what to do with it. Devices that already have their firmware configured will tell the OS what to do with it, usually through the Human Interface Device (HID) standard. So I need to ask the OS where it put the file. +

$ lsusb
+Bus 003 Device 022: ID 28de:1102 Valve Software Wired Controller

+ This gives us the Bus number, Device number, and both the Device ID and Vendor ID. The bus and device number are abstractions for userspace created by udev. In this instance they seem very redundant and abstract away from what I'm trying to accomplish. What I am trying to do is take the raw input from the device at a kernel level, so I need to ask udev where it's getting this device from. +

$ udevadm info -q path -n /dev/bus/usb/003/022
+/sys/devices/pci0000:00/0000:00:14.0/usb3/3-7

+ This shows us where the kernel has put the device. We can go into this series of directories and find it. Problem solved! Almost. +

+ +

+ In this case, with the steam controller, it actually has 3 subdevices, each with their own hidraw file. As it turns out, the controller can also be used in place of a mouse and keyboard. One could poke around the files some more to differentiate them, but I found a better solution. +

sudo usbhid-dump -m 28de:1102 -ed > descriptor.txt
+grep -A 10 "DESCRIPTOR" descriptor.txt | grep -v "DESCRIPTOR" | grep -v "STREAM" | tr -d ' \n' > descriptor_hex.txt
+rexx rd.rex -d --hex "$(cat descriptor_hex.txt)"

=== /dev/hidraw6 ===
+Name:             Valve Software Wired Controller
+Vendor:Product:   0x28de:0x1102
+Bus type:         3
+Physical:         usb-0000:00:14.0-7/input0
+Descriptor size:  65 bytes
+Descriptor (first 32 bytes): 05 01 09 06 95 01 A1 01 05 07 19 E0 29 E7 15 00 25 01 75 01 95 08 81 02 95 01 75 08 81 01 95 05 
+Type hints:       [KEYBOARD] 
+
+=== /dev/hidraw7 ===
+Name:             Valve Software Wired Controller
+Vendor:Product:   0x28de:0x1102
+Bus type:         3
+Physical:         usb-0000:00:14.0-7/input1
+Descriptor size:  52 bytes
+Descriptor (first 32 bytes): 05 01 09 02 A1 01 09 01 A1 00 05 09 19 01 29 05 15 00 25 01 95 05 75 01 81 02 95 01 75 03 81 01 
+Type hints:       [MOUSE] 
+
+=== /dev/hidraw8 ===
+Name:             Valve Software Wired Controller
+Vendor:Product:   0x28de:0x1102
+Bus type:         3
+Physical:         usb-0000:00:14.0-7/input2
+Descriptor size:  33 bytes
+Descriptor (first 32 bytes): 06 00 FF 09 01 A1 01 15 00 26 FF 00 75 08 95 40 09 01 81 02 95 40 09 01 91 02 95 40 09 01 B1 02

+ +

+ I found a tool called RDD! HID Report Descriptor Decoder on github by abend0c1. It takes the data from usbhid-dump and performs lookups on all the devices given. It's a very comprehensive tool that gave much more output than I listed. +

Operational Transformation

+ The core algorithm that makes collaboration possible is operational transformation (OT). When two users + edit different parts of the document, their operations must be transformed relative to each other to + maintain consistency. +

+ +

+ "Operational transformation is the mathematical foundation that allows distributed systems to converge + to the same state despite concurrent modifications." +

+ +

+ Our implementation uses a three-operation model: insert, delete, and retain. Each operation carries + positional information that gets transformed when concurrent operations occur. +

+ +

Key Features

+ The final implementation includes several sophisticated features that enhance the collaborative experience: +

+ +

Cursor Tracking: Real-time display of where other users are typing
Presence Indicators: Shows who's currently viewing the document
Conflict-Free Resolution: Automatic handling of concurrent edits
Undo/Redo: Full history management that works across collaborative sessions
Rich Text Formatting: Support for bold, italic, lists, and more

+ +

Performance Optimizations

+ To ensure smooth performance even with large documents and many concurrent users, several optimizations + were implemented: +

+ +

Operation batching to reduce network overhead
Delta compression for efficient data transmission
Lazy loading of document history
Client-side caching with service workers

+ +

Code Implementation

+ The operational transformation logic is implemented in JavaScript. Here's a simplified example of how + operations are transformed when they conflict: +

+ +

function transformOperation(op1, op2) {
+  // If operations affect different positions, no transformation needed
+  if (op1.position < op2.position) {
+    return op1;
+  }
+  
+  // Transform op1 based on op2s changes
+  if (op2.type === 'insert') {
+    return {
+      ...op1,
+      position: op1.position + op2.length
+    };
+  }
+  
+  if (op2.type === 'delete') {
+    return {
+      ...op1,
+      position: Math.max(op1.position - op2.length, op2.position)
+    };
+  }
+  
+  return op1;
+}

+ +

WebSocket Event Handler

+ The server-side WebSocket handler manages incoming operations and broadcasts them to all connected clients: +

+ +

io.on('connection', (socket) => {
+  console.log('Client connected:', socket.id);
+  
+  socket.on('join-document', (docId) => {
+    socket.join(docId);
+    socket.to(docId).emit('user-joined', {
+      userId: socket.id,
+      timestamp: Date.now()
+    });
+  });
+  
+  socket.on('operation', (data) => {
+    const { docId, operation, version } = data;
+    
+    // Transform and broadcast operation
+    const transformed = applyTransformation(operation, version);
+    socket.to(docId).emit('operation', transformed);
+    
+    // Save to database
+    saveOperation(docId, transformed);
+  });
+});

+ +

React Component Example

+ On the client side, React components manage the editor state and handle user input: +

+ +

const CollaborativeEditor = () => {
+  const [content, setContent] = useState('');
+  const [cursors, setCursors] = useState({});
+  const wsRef = useRef(null);
+  
+  useEffect(() => {
+    // Connect to WebSocket server
+    wsRef.current = io('wss://api.example.com');
+    
+    wsRef.current.on('operation', (op) => {
+      setContent(prev => applyOperation(prev, op));
+    });
+    
+    wsRef.current.on('cursor-move', (data) => {
+      setCursors(prev => ({
+        ...prev,
+        [data.userId]: data.position
+      }));
+    });
+    
+    return () => wsRef.current.disconnect();
+  }, []);
+  
+  return (
+    <div className="editor">
+      <textarea 
+        value={content}
+        onChange={handleChange}
+        onSelect={handleCursorMove}
+      />
+      {renderCursors(cursors)}
+    </div>
+  );
+};

+ +

Lessons Learned

+ Throughout this project, several important insights emerged about building real-time collaborative systems: +

+ +

+ Network Reliability: Never assume the network is stable. Implementing robust reconnection + logic and conflict resolution for offline edits proved essential for a production-ready system. +

+ +

+ State Management: Keeping client and server state synchronized requires careful attention + to edge cases. The operational transformation algorithm must handle not just simple concurrent edits, but + also complex scenarios involving multiple clients making rapid changes. +

+ +

+ User Experience: The technical implementation, no matter how elegant, means nothing if + users experience lag or data loss. Optimizing for perceived performance was as important as actual performance. +

+ +

Future Enhancements

+ While the current implementation is robust, several exciting enhancements are planned for future iterations: +

+ +

End-to-end encryption for sensitive documents
Voice and video integration for enhanced collaboration
AI-powered suggestions and auto-completion
Mobile app support with native performance
Advanced permission systems for enterprise deployments

+ +

+ This project demonstrates that building truly collaborative software requires deep understanding of + distributed systems, careful attention to user experience, and thoughtful architecture decisions. The + result is a system that feels magical to users while being built on solid engineering principles. +

The Challenge

+ Building a real-time collaborative editor presents unique technical challenges. When multiple users edit + the same document simultaneously, conflicts arise that must be resolved elegantly without losing data or + disrupting the user experience. This project explores operational transformation algorithms and their + implementation in a modern web application. +

+ +

+ The goal was to create an editor that feels as responsive as Google Docs while maintaining data consistency + across all connected clients. Every keystroke needed to be transmitted, transformed, and applied in real-time + without introducing latency or conflicts. +

+ +

Technical Architecture

+ The system is built on a microservices architecture with distinct components handling different aspects + of the collaborative experience: +

+ +

WebSocket Server

+ At the heart of the system is a Node.js WebSocket server that manages all real-time connections. Each + document session creates a room where clients can join, and all operations are broadcast through this central hub. +

+ +

Handles client connections and disconnections gracefully
Broadcasts operations to all connected clients in a room
Implements heartbeat mechanisms to detect stale connections
Scales horizontally using Redis pub/sub for multi-server deployments

+ +

Operational Transformation

+ +

+ "Operational transformation is the mathematical foundation that allows distributed systems to converge + to the same state despite concurrent modifications." +

+ +

+ Our implementation uses a three-operation model: insert, delete, and retain. Each operation carries + positional information that gets transformed when concurrent operations occur. +

+ +

Key Features

+ The final implementation includes several sophisticated features that enhance the collaborative experience: +

+ +

Cursor Tracking: Real-time display of where other users are typing
Presence Indicators: Shows who's currently viewing the document
Conflict-Free Resolution: Automatic handling of concurrent edits
Undo/Redo: Full history management that works across collaborative sessions
Rich Text Formatting: Support for bold, italic, lists, and more

+ +

Performance Optimizations

+ To ensure smooth performance even with large documents and many concurrent users, several optimizations + were implemented: +

+ +

Operation batching to reduce network overhead
Delta compression for efficient data transmission
Lazy loading of document history
Client-side caching with service workers

+ +

Code Implementation

+ The operational transformation logic is implemented in JavaScript. Here's a simplified example of how + operations are transformed when they conflict: +

+ +

function transformOperation(op1, op2) {
+  // If operations affect different positions, no transformation needed
+  if (op1.position < op2.position) {
+    return op1;
+  }
+  
+  // Transform op1 based on op2's changes
+  if (op2.type === 'insert') {
+    return {
+      ...op1,
+      position: op1.position + op2.length
+    };
+  }
+  
+  if (op2.type === 'delete') {
+    return {
+      ...op1,
+      position: Math.max(op1.position - op2.length, op2.position)
+    };
+  }
+  
+  return op1;
+}

+ +

WebSocket Event Handler

+ The server-side WebSocket handler manages incoming operations and broadcasts them to all connected clients: +

+ +

io.on('connection', (socket) => {
+  console.log('Client connected:', socket.id);
+  
+  socket.on('join-document', (docId) => {
+    socket.join(docId);
+    socket.to(docId).emit('user-joined', {
+      userId: socket.id,
+      timestamp: Date.now()
+    });
+  });
+  
+  socket.on('operation', (data) => {
+    const { docId, operation, version } = data;
+    
+    // Transform and broadcast operation
+    const transformed = applyTransformation(operation, version);
+    socket.to(docId).emit('operation', transformed);
+    
+    // Save to database
+    saveOperation(docId, transformed);
+  });
+});

+ +

React Component Example

+ On the client side, React components manage the editor state and handle user input: +

+ +

const CollaborativeEditor = () => {
+  const [content, setContent] = useState('');
+  const [cursors, setCursors] = useState({});
+  const wsRef = useRef(null);
+  
+  useEffect(() => {
+    // Connect to WebSocket server
+    wsRef.current = io('wss://api.example.com');
+    
+    wsRef.current.on('operation', (op) => {
+      setContent(prev => applyOperation(prev, op));
+    });
+    
+    wsRef.current.on('cursor-move', (data) => {
+      setCursors(prev => ({
+        ...prev,
+        [data.userId]: data.position
+      }));
+    });
+    
+    return () => wsRef.current.disconnect();
+  }, []);
+  
+  return (
+    <div className="editor">
+      <textarea 
+        value={content}
+        onChange={handleChange}
+        onSelect={handleCursorMove}
+      />
+      {renderCursors(cursors)}
+    </div>
+  );
+};

+ +

Lessons Learned

+ Throughout this project, several important insights emerged about building real-time collaborative systems: +

+ +

+ Network Reliability: Never assume the network is stable. Implementing robust reconnection + logic and conflict resolution for offline edits proved essential for a production-ready system. +

+ +

Future Enhancements

+ While the current implementation is robust, several exciting enhancements are planned for future iterations: +

+ +

End-to-end encryption for sensitive documents
Voice and video integration for enhanced collaboration
AI-powered suggestions and auto-completion
Mobile app support with native performance
Advanced permission systems for enterprise deployments

+ +

Steam Controller Driver for Actual Motors

Backstory

Flight Simulator

WebSocket Server

Physical Applications

Parsing the Byte Stream

Operational Transformation

Key Features

Performance Optimizations

Code Implementation

WebSocket Event Handler

React Component Example

Lessons Learned

Future Enhancements

Building a Real-Time Collaborative Editor

The Challenge

Technical Architecture

WebSocket Server

Operational Transformation

Key Features

Performance Optimizations

Code Implementation

WebSocket Event Handler

React Component Example

Lessons Learned

Future Enhancements

BLOG

Steam Controller Driver For Actual Motors

Air-Quality Monitoring and Data Collection

Home Sweet Homelab

Automating Automation with n8n

Eric Steen

Professional Summary

Experience

Site Reliability Engineer

Hardware Engineer

Skills

Projects

Project Name