Fileber

Appearance

The choice of networking protocol ​

The first choice I had to make is which networking protocol to use.

Because Fileber needs to work with most modern browsers (so that it is easily accessible), the available choices are:

  • Plain HTTP requests
  • Server-Sent Events (SSE)
  • WebSocket
  • WebRTC
  • Bluetooth

Plain HTTP requests ​

Plain HTTP requests incur higher overhead because they require sending HTTP headers for every data chunk.
Polling cost and polling latency are also worse than other approaches.

Server-Sent Events (SSE) ​

SSE would mitigate the polling cost and polling latency.
However, SSE only allows sending UTF-8 data. Thus, this method still incurs encoding/decoding overhead.

WebSocket ​

WebSocket provides lower overhead and lower latency, while also supporting transferring binary data.
However

  • It requires that the devices (clients) must be connected to the internet in order to connect to Filber WebSocket Server.
  • It requires that data must be relayed through Fileber WebSocket server
    • This incurs internet costs to both Fileber Server and the user.
    • Some users might not trust Fileber Server and concerned about their data security/privacy.

WebRTC ​

WebRTC allows using the same programming interface to connect and transfer data in different network setups. For example, given that we want to transfer data between 2 devices,

  1. If both devices are using the same local network, it allows those devices to transfer data to each other directly.

  2. If one of the 2 devices is behind NAT, it can use a STUN server and still allows those devices to transfer data to each other directly.

  3. If both devices are behind NAT (symmetric NAT), it uses a TURN server and allows those devices to transfer data to each other through the TURN server as a relay.

Such capability makes it fast and costless to transfer data in cases number 1 and 2, where data does not need to go through Fileber Server.

However, the 3rd case still requires a TURN server to relay the data, similar to the WebSocket setup. The TURN server can be hosted by Fileber or by 3rd party services, but it still incurs network costs to Fileber.

Bluetooth ​

Bluetooth can support the case of devices that are physically close to each other, but not when the devices are distanced.
Additionally, Web Bluetooth API is currently only supported by Chrome.

The Decision ​

WebRTC has the most capabilities and hence can support all use cases.
However

  • Existing implementations of TURN and SFU servers than I can find don't provide enough control or observability that I want. On the other hand, I don't think it makes sense to spend money for metered TURN services at this point.
  • While WebRTC has the capabilities to support 3 network scenarios that I mentioned above, I currently don't have the time nor expertise to test and debug all those scenarios properly. I'm also not sure whether the support for WebRTC is as stable as WebSocket on all devices (with different OS and browser combinations).

Decision

Thus, I decided that I don't want to spend too much resources to make WebRTC work for Fileber right now, and that I should make the quickest implementation possible to validate my product idea first. I chose WebSocket for now because it is quick to implement and most devices are connected to the internet nowadays.

Later on, when the product is shown to be useful, I can invest more into the networking technology, such as

  • Implementing my own TURN and SFU server and migrating to WebRTC
  • Using a hybrid of WebSocket and WebRTC
  • Using a hybrid of WebSocket and Bluetooth