Lightweight abstraction for efficient management of scalable file systems

What is ActiveFS?

ActiveFS is a lightweight asynchronous Java library that provides a tiny abstraction on top of common file operations. It provides upload, download, append, list, copy, move, delete, and other methods for operating with local, remote or distributed storage.

Where ActiveFS can be used?

There are several ActiveFS implementations:
  • Local ActiveFS designed for working with file systems located on a single machine. Ideal to implement a local file storage for your application, for example database storage, backup data storage, local logs storage, etc.
  • Client/Server implementations for operating with files that are stored remotely. This implementation supports two protocols:
    • Custom TCP-based binary protocol with zero overhead. Resembles FTP, but simpler and more efficient.
    • HTTP protocol that allows to define a REST API. It also provides third-party clients like browsers access remote ActiveFS servers.
  • Cluster ActiveFS is designed for distributed big data workload. It's optimized for processing large immutable datasets or append-only files. Cluster ActiveFS was intentionally made not POSIX-compilant and works only with immutable files. In this way the whole system is simple, lightweight and resilient to server or network failures. Also, this approach allows to avoid the overhead of master-slave architecture and synchronization of mutable files. With ActiveFS you can create distributed, scalable P2P file systems with built-in support for rebalancing, failover, and scalability.
  • ActiveFS Adapters enable filtering, mounting, transforming file names, adding and removing prefixes, subdirectories, etc.

Efficiently work with distributed file storage

ActiveFS is an efficient and lightweight alternative for kernel space distributed FS like Ceph or Apache Hadoop. ActiveFS is like IPFS on the scale of a single data center It allows to create combined, fault-tolerant cluster NFS. Moreover, ActiveFS contains a set of predefined communication strategies that can be simply combined.
private RemoteFsClusterClient client;

    ...

Eventloop eventloop = Eventloop.create();

Map<Object, FsClient> clients = ...;

client = RemoteFsClusterClient.create(eventloop, clients)
    .withReplicationCount(N / 2)
    .withServerSelector(RENDEZVOUS_HASH_SHARDER);

Add ActiveFS to your project

You can add ActiveFS to your project by importing its Maven repository.
These docs cover the most recent release of ActiveFS, which is v2.2 published Sep 11, 2020 (see on GitHub) .

Project background

ActiveFS is one of the ActiveJ technologies that also include lightning-fast serializers, bytecode generators, dependency injection tools and others. To learn more about ActiveJ technologies, visit ActiveJ website.