Machine learning solutions

How to aggregate and manage data from various sources and formats

Need to gather external datasets or databases; Data is distributed across multiple locations, collected from different data sources (across data centers, clouds, and edges), and converted into a unified format; Massive unstructured or semi-structured data (images, videos, audio, annotation files, etc.) require high storage throughput and latency.

Traditional storage cannot meet the requirements of different stages of machine learning for storage capacity, performance, and interface protocols

The data collection and archiving stage has typical I/O intensive characteristics, requiring high bandwidth and large capacity; The model training phase involves a large number of random and small file read operations, requiring high bandwidth and low latency; The inference stage requires low latency and high performance; The data archiving and preparation stage requires high management capabilities for storing and retrieving massive amounts of data, with object storage (S3 protocol) being more suitable. However, training inference requires high storage latency response and concurrent access capabilities, and due to the historical reasons of the training platform, it is usually more suitable to use distributed file systems (NFS/CIFS, POSIX interface protocol).

The utilization rate of expensive GPU resources is not high, and resources cannot be shared

In current machine learning solutions, it is increasingly common to use GPUs to provide computing power to accelerate the learning and training process. Expensive GPU resources are shared, and multi machine and multi card clusters can simultaneously perform more training tasks, which not only accelerates the learning process but also improves resource utilization and reduces resource waste.

Support the aggregation and massive storage of unstructured, semi-structured, and structured data

Tag retrieval, intelligent data management

Multi access protocol support

Client distributed cache acceleration, unleashing GPU potential