24 Jul 2024

Groundbreaking Edge AI server

Edge AI Server Solutions for Generative AI

Advantech has announced a groundbreaking Edge AI server solution for generative AI, featuring Phison’s patented aiDAPTIV+ technology and says the AIR-520 Edge AI Server, powered by an AMD EPYC 7003 series processor, integrates SQ ai100 AI SSDs, NVIDIA RTX GPU cards, an Edge AI SDK, and NVIDIA AI Enterprise to provide a ready-to-deploy solution.

The company says generative AI tools such as large language models (LLMs) are transforming enterprise knowledge management by automating data organisation, retrieval, and analysis, thereby boosting productivity and improving decision-making. Custom LLMs enhance accuracy, while edge training increases data privacy, though it can be more costly. This solution supports LLM fine-tuning with 1-4 GPU cards and SQ ai100 AI SSDs, enabling enterprises to train LLMs cost-effectively while keeping sensitive data secure at the edge.

Advantech says it offers four options: AIR-520-L13B/L33B/L70B, and L70B-Plus, tailored for different scales and applications. The L13B is ideal for real-time applications such as chatbots and language translation. The L33B is suited for more complex tasks, enhancing productivity and innovation in content creation. The L70B excels in sophisticated data analysis and decision-making for specialised domains. Additionally, the L70B-Plus, equipped with the NVIDIA AI Enterprise software platform, offers end-to-end, reliable and optimised AI SDKs with long-term support and expert consulting services, ensuring efficient deployment of business applications.

All solutions include SQ ai100 AI SSDs, which leverage Phison’s aiDAPTIV+ technology. These SSDs act as an extension of GPU vRAM, enabling the system to fine-tune LLMs with minimal GPU cards. This approach not only eases the budget barrier, but also makes the Edge AI Server more compact compared to traditional large rack-mount servers. The AIR-520 Edge AI Server has been designed for use in a diverse range of edge AI applications. Its size is comparable to a desktop PC, and it can be rack-mounted with the appropriate accessories. Its low profile allows for easy deployment of an edge AI fine-tuning environment, eliminating concerns about space and maintenance.

In addition to LLM fine-tuning capabilities, Advantech says it provides an Edge AI SDK with the GenAI Training Studio, preloaded with Llama-2 13B/33B/70B models for applications like chatbots and data analysis. This simplifies and accelerates customer-specific LLM model training and inference evaluation on the AIR-520. Furthermore, Advantech’s DeviceOn provides OTA software/container updates and remote management, facilitating efficient edge AI orchestration and long-term maintenance.

For customers requiring customised solutions, Advantech says its European DMS team offers a wide range of local design and manufacturing services.



Company info: Advantech