5 posts tagged with "Tech"

To effectively implement LLM services, a robust LLMOps framework is essential. Among its components, observability (o11y) has evolved beyond simple monitoring to become a critical enabler of the system’s meta-intelligence.

---

The Evolution of o11y into Meta-Intelligence​

Early LLM o11y focused on collecting metrics such as token usage, response time, response content, and user feedback to monitor performance. We adopted Langsmith, a commercial tool, to monitor the execution process of AI logic. Later, we integrated Langfuse, an open-source tool, allowing our organization to selectively use either tool based on licensing requirements.

However, as the number of AI Agent service users grew, it became clear that accumulated data could no longer provide meaningful insights through simple log analysis. Consequently, we decided to transform o11y data from mere "observation logs" into a meta-intelligence tool. This system leverages AI Agent outputs and user feedback to automatically reformulate questions or enhance response quality by adjusting model behavior.

In essence, o11y data transcends real-time performance monitoring to become the cornerstone of a feedback loop that enables AI Agents to self-improve.

Academically, this approach aligns with the growing focus on AgentOps or Agentic AI observation systems. There is a movement to propose comprehensive observation frameworks for AgentOps, tracking various artifacts such as execution paths, internal logic, tool calls, and planning stages. Beyond black-box evaluations, the importance of inferring and optimizing behavioral patterns based on agent execution logs is increasingly emphasized.

---

Mellerikat’s ‘Meta-Intelligence o11y’ Model Architecture​

Our o11y meta-intelligence strategy is structured as follows:

1. Trace-Based Execution Logging Using Langsmith and Langfuse, we capture LLM calls, internal planning, token usage, responses, and feedback at the trace session level.

2. Real-Time Anomaly Detection and Feedback Loop When issues like latency or hallucinations are detected, alerts and automated reports are generated instantly, with an automatic tagging system triggered based on user feedback.

3. Meta-Layer Inference Engine Based on collected data, the system provides intelligent suggestions, such as "Which questions are more appropriate?" or "How should responses be restructured for better effectiveness?"

4. Automated Prompt Tuning and Deployment The meta-intelligence engine automatically refines prompts, and updated logic is applied in real-time through a CI/CD pipeline.

This architecture moves beyond manual tuning of LLM logic, aiming for meta-intelligence where the system continuously proposes and implements improvements.

Today, LLM-based AI Agents are evolving from mere response generators into agentic AI, capable of autonomous reasoning, planning, and execution. In this context, o11y transcends monitoring to become the core infrastructure for self-learning meta-intelligence. Ultimately, o11y is the key to transitioning beyond basic observability into a paradigm where AI Agents continuously evolve, enabling sustained learning Agent services.

For more details on Mellerikat o11y, refer to this link.

Background​

Meraki’s Cloud-Managed Service already boasts an exceptional infrastructure. If a variety of third-party apps, particularly AI-based services, could seamlessly integrate with this cloud platform, the true potential for enhancing Meraki’s value could be realized.

Currently, Meraki Cloud includes an App Store with some available apps, but it faces clear limitations:

• Integration with Meraki Cloud Services
  • App installation and deployment are restricted. Only select partners can officially register apps, and the installation process is complex or not automated.
  • Third-party apps are not fully integrated with the Meraki Dashboard, leading to fragmented user experiences or dispersed management points.
  • Limitations in APIs and SDKs hinder sufficient integration and scalability with external services.

Upgrading Meraki Cloud to the next level and establishing best practices for a third-party app ecosystem make the integration of Meraki Smart Camera with mellerikat EVA a highly significant case study.

---

Understanding Meraki Smart Camera & Cloud​

Meraki Smart Camera and Cloud form an outstanding service combination. Far from being a simple CCTV, Meraki Camera aims to be a smart video platform organically integrated with network and security infrastructure.

Key Features of Meraki Camera​

• All-in-One Structure Meraki Cameras have built-in storage (SSD or SD card), eliminating the need for a separate Network Video Recorder (NVR), and support Power over Ethernet (PoE) for power and network connectivity.

• Cloud-Based Management All configuration, operation, and maintenance are centrally managed through the Meraki Dashboard (web-based UI), enabling real-time monitoring and configuration changes without on-site visits.

• Intelligent Analytics Features like motion-based event detection, person detection, and area-specific analytics (e.g., heatmaps, dwell time) provide insights beyond basic surveillance for spatial operations.

• End-to-End Security Stored video data is encrypted during transmission and managed under Cisco Meraki’s security framework, with user role-based access control and audit logs.

---

Core Value of Meraki Camera Cloud​

• Centralized Management Cameras installed worldwide can be managed from a single Meraki Cloud, ideal for large-scale, multi-site enterprises or global chains.

• Instant Video Access While videos are stored locally in the camera, they can be streamed quickly via the cloud anytime, anywhere, with fast and intuitive event or time-based searches.

• Smart Search and Analytics Timeline-based motion searches, person detection-based searches, and cross-camera event tracking significantly reduce the time needed to review footage.

• Hardware Maintenance Efficiency Camera firmware updates are applied automatically, and alerts for anomalies or failures enable proactive responses.

---

True Smart Meraki Camera​

Despite this robust infrastructure, it’s time to consider the next step: delivering True Smart value to customers.

• Cameras should recognize and deeply understand a wider range of scenarios.
• They should enable user-customized scenario recognition and automatically execute desired follow-up actions based on recognized information.
• Data movement, storage, and utilization systems should be optimized to enhance cost efficiency.

These processes should be easily managed and controlled using Large Language Models (LLMs).

For example, cameras could automatically detect specific events (e.g., hazardous situations, congestion, VIP visits) and trigger actions like notifications, automatic door opening, or lighting control. AI could also recommend or optimize video data storage and transfer policies. With an LLM-based natural language interface, queries like “Tell me about the customer who stayed the longest in the store today” can be handled effortlessly.

---

mellerikat EVA + Meraki Camera​

To realize the True Smart vision for Meraki Camera, we propose mellerikat EVA, a Vision AI solution offering the following services.

Features and User Service Scenarios of Meraki + EVA​

• Real-Time Advanced AI Analytics EVA analyzes video feeds from Meraki Cameras in real time, providing services like person, vehicle, and object recognition, behavior analysis, and hazard detection.

• Cloud-Based Seamless Deployment AI capabilities can be activated instantly through integration with Meraki Cloud, without additional hardware.

• Customized User Services Users can easily define desired events (e.g., intrusions in specific areas, queue length, customer behavior patterns), and EVA automatically recognizes and provides alerts for them.

• Demo Scenario For instance, when store congestion increases, EVA can automatically notify the manager and trigger synchronized lighting or audio announcements, enabling immediate operational applications.

---

mellerikat EVA​

EVA is a powerful technical platform for AI services, integrated with various mellerikat technology components.

Technical Strengths of the Backend​

• Modular AI Engine Supports flexible combinations of AI modules for object detection, behavior recognition, face/license plate recognition, and more.

• Continuous High-Quality Service Easily deploys and utilizes new models to consistently deliver high-performance services.

• Auto-Scaling and Distributed Processing Built on a cloud-native architecture, it automatically scales resources up or down based on video analysis workloads.

• API/SDK-Based Scalability Offers RESTful APIs, webhooks, and SDKs for seamless integration with external systems (ERP, POS, IoT, etc.).

• No-Code/Low-Code Interface Provides an intuitive UI, allowing non-experts to easily design and manage AI analysis scenarios.

---

The Importance of mellerikat EVA and Meraki Collaboration​

EVA could partner directly with various camera vendors to provide services, operate as a cloud-based SaaS, or be offered through white-labeling with camera hardware vendors. It could also be installed in a vendor’s cloud or on-premises environment.

However, these approaches face challenges in achieving economies of scale during initial service setup and operations.

• Meraki already has a globally scaled cloud infrastructure and a large customer base.
• Combining Meraki Cloud with EVA enables immediate large-scale service launches and expansion without additional infrastructure.
• Leveraging Meraki’s App Store and API ecosystem automates EVA’s deployment, operation, and updates, maximizing customer accessibility.
• This collaboration eliminates cost barriers to adopting AI in cameras and positions Meraki and EVA to lead the global standard for Vision AI services.

---

Beyond AI​

EVA aims to grow beyond providing AI functions into a technical platform that unlocks diverse possibilities through cameras (Vision).

• Users can train cameras to recognize desired scenarios.
• Recognized scenarios can trigger subsequent actions via the Meraki Control Plane (MCP).
• These processes are effectively managed on an integrated technical platform.

Human behavior can be simplified as “recognition → action,” occurring across physical channels on the organically connected platform of the human body. EVA is evolving to technically replicate such an organic platform.

---

EVA’s Vision: Becoming the AWS of the Smart Camera Market​

EVA aims to transform the on-premises-centric CCTV market into a cloud-based service market. It envisions a future where numerous Vision sensors operate smartly on an interconnected platform, enabling sci-fi-inspired recognition-action scenarios to be implemented and integrated as services.

Specialized Intelligence: The Key to Business Innovation Beyond General Intelligence​

Since the digital revolution, artificial intelligence (AI) has rapidly advanced, bringing transformative changes to our daily lives and industries. The emergence of Generative AI (Gen AI) has made AI technology accessible to everyone, but it has also introduced various challenges. While a universal AI capable of excelling in all domains is an ideal goal, in reality, specialized intelligence tailored to specific fields often delivers greater value.

Generative AI exhibits characteristics close to General Intelligence (AGI), enabling it to perform a wide range of tasks. This Gen AI capability contrasts with AI technologies specialized for specific problems, offering versatility across diverse contexts and scenarios.

In the field of AI, Artificial General Intelligence (AGI) is progressing at an impressive pace. While this development is remarkable, pursuing the highest level of general intelligence can sometimes seem overly ambitious and impractical. Instead, true value lies in developing and leveraging domain-specific intelligence to address challenges within particular fields, embodying specialized expertise and capabilities tailored to specific tasks and contexts.

Specialized intelligence technologies provide optimized models for specific industries and use cases, enabling enterprises to effectively tackle complex challenges. Unlike general AI, specialized intelligence deeply understands the unique data and patterns of a specific domain, delivering superior results. Over time, this approach enables higher-quality predictions and analyses, maximizing business value. Mellerikat supports the seamless development and operation of such specialized intelligence, empowering enterprises to effectively leverage expert knowledge.

Large Language Models (LLMs) can collect and summarize diverse business-related news data to provide information on specific topics. However, predicting whether copper prices will rise or fall exceeds the capabilities of LLMs. This task requires domain-specific intelligence, such as a predictive machine learning (ML) engine integrated with the expertise of copper pricing specialists.

---

The Role and Advantages of Domain-Specific AI Engines​

Integration of Expert Knowledge​

• Tailored Solution Development and Incorporation of Accumulated Experience: Developing domain-specific intelligence requires specialized expertise and modeling experience. Mellerikat provides AI Contents and the ALO Framework to seamlessly integrate this knowledge into AI engines, enabling enterprises to leverage AI solutions optimized for their specific business models.

Enhanced Data Security and Privacy​

• Selective Data Collection: By processing data through specialized intelligence before sending it to generative AI, the risk of sensitive data leaks is minimized, which is critical for protecting enterprise information.

Increased Cost Efficiency​

• Cost Reduction: Utilizing filtered data reduces unnecessary computations, lowering costs and enabling efficient use of computing resources.
• Computing Resource Optimization: Optimizes resources to maintain high performance while achieving cost efficiency.

Improved Reliability and Accuracy​

• Hallucination Prevention: Specialized intelligence analyzes and validates data, reducing the likelihood of unrealistic responses, which enhances AI system reliability.
• Decision-Making Support: Provides accurate and reliable information to support critical decision-making, minimizing business risks and maximizing outcomes.

---

Enhancing Expertise Through the Integration of Specialized Intelligence and LLMs​

Specialized AI engines leverage domain intelligence analysis results, applying prompt engineering and Relevance Augmented Generation (RAG) techniques to enable LLMs to deliver more professional and efficient responses. Mellerikat facilitates this integration, empowering enterprises to utilize AI technology more effectively.

• Prompt Engineering: Designs sophisticated prompts to deliver key insights analyzed by specialized intelligence to LLMs, enabling the generation of professional responses tailored to specific domains rather than generic answers.
• Relevance Augmented Generation (RAG): Integrates the results of specialized intelligence into the LLM’s learning process, ensuring that generative models produce more accurate and reliable responses.

This approach transcends the limitations of using generative AI solely for search, enabling the creation of valuable responses that directly support business decision-making.

---

The true power of AI lies not in pursuing general intelligence alone but in the strategic application of specialized intelligence. Mellerikat supports the development and operation of such intelligence, serving as a critical enabler for enterprises to maintain sustainable competitiveness. By combining general and specialized intelligence, AI technology can be leveraged more effectively in business contexts.

Building an MLOps Platform with Data Security in Mind​

While artificial intelligence (AI) is driving transformative changes across various industries, concerns about data security remain a persistent challenge during implementation. Enterprises often handle sensitive information, and the risk of data leaks or misuse poses a significant barrier. As a result, many organizations hesitate to adopt AI technologies, with data security issues being a primary obstacle.

However, there is a solution to these concerns: the Mellerikat MLOps platform. Mellerikat combines data security with AI technology to enable enterprises to innovate safely. It minimizes outgoing data and uses only necessary data to operate AI models.

Mellerikat’s approach prioritizes data security while opening up diverse possibilities for enterprises to leverage AI technologies to enhance their competitiveness.

This article explores why data security is critical when adopting AI and how Mellerikat addresses these concerns. We will highlight the strengths of Mellerikat in delivering optimized AI solutions while maintaining data security and discuss the positive impact this can have on your business.

---

Optimized Service Operations with Private Networks​

Many enterprises are concerned about transmitting data from private network environments to external systems. To address this, Mellerikat offers the following optimized service models:

• Edge App for Model Servicing: Installed within the customer’s private network where data resides, enabling immediate data access.
• Edge Conductor: Deployed on-site but capable of communicating with the private network and connecting to external systems via a DMZ.

This structure allows the Edge Conductor and Edge App to be installed on-site, ensuring that no data leaves the premises while operating AI models.

---

Secure Data Training and Inference Processes​

For model training, only selected datasets are transmitted from the Edge Conductor to external systems. These selectively transmitted datasets can be periodically used for training. Once training is complete, the AI Conductor provides the inference model file to the Edge Conductor. The data sent to the Edge Conductor consists solely of this model file, and cloud data used for training can be deleted to ensure temporary retention.

Additionally, when registering an AI solution, the training data can be registered alongside it, allowing the model to be deployed and operated without sending Edge Conductor data externally. Alternatively, solutions that perform both training and inference within the Edge App can be deployed to operate the model.

---

Services Across Diverse Environments​

AI Model Services in Multi-Cloud Environments​

Enterprises store data across various cloud platforms such as AWS, GCP, and Azure. To utilize AI models while maintaining data security, the models must be executed within the respective cloud. By installing the Edge App on each cloud’s computing resources and, if needed, deploying the Edge Conductor, models can be serviced effectively.

AI Model Services on On-Premises Servers​

For enterprises with policies that prohibit storing data in the cloud, data can be maintained in on-premises data centers or servers. In such cases, the Edge App and Edge Conductor can be installed on these servers to leverage AI models.

AI Model Services in Factories/Local Computers​

In environments like factories, AI models can be used on computers connected to production lines for tasks such as inspections or predictive maintenance, typically operating within isolated factory networks for security. By installing the Edge App on these computers and deploying the Edge Conductor on an on-site server, AI models can be serviced. Dozens of computers can run Edge Apps for real-time inference, while the Edge Conductor continuously operates models that learn from on-site changes.

---

Inside Mellerikat​

Mellerikat’s MLOps platform consists of the AI Conductor, which manages solutions and oversees AI model training, and the Edge Conductor and Edge App, which provide model management and inference services. This structure enables model servicing across diverse environments while maintaining data security. Unlock new possibilities with Mellerikat’s platform.

For more details on secure services, refer to this link.

Mellerikat Service Models​

Mellerikat is a powerful tool for digital growth and innovation, combining AI and cloud technologies to optimize enterprise business processes. Through collaboration with various partners, it delivers tailored solutions to enhance customer competitiveness.

The Mellerikat platform is highly flexible, enabling partners to apply it across diverse service models. Each customer can receive services tailored to their specific needs and circumstances, accelerating their digital transformation (DX). This article explores these service models, their operations, and the benefits they provide in detail.

To use Mellerikat, a basic cloud infrastructure is required. The AI Conductor is installed on this infrastructure to manage AI solutions, train data, and generate models. Partners leverage this cloud infrastructure to configure services and deliver solutions aligned with customer requirements.

---

Service Models​

Report Service​

This service involves deploying all components on the partner’s cloud and reporting the results of AI models. For example, it provides solutions for raw material price forecasting or anomaly detection and root cause analysis based on customer-provided data. Customers can receive essential reports without building their own infrastructure, enabling data-driven decision-making.

Solution Service​

In this model, partners handle MLOps for the AI model, while customers access services via the Edge App. Customers do not need to manage model maintenance; they simply input data into the Edge App to receive real-time AI analysis results. The Edge App can be installed on local computers or existing systems for quick and seamless service access.

Solution Service (Security / Operation)​

For customers requiring data security and direct model operation, the Edge Conductor and Edge App are deployed on the customer’s infrastructure. Partners provide solutions through the AI Conductor to address customer challenges, while customers manage their data to create and operate models. This minimizes data breach risks and allows customers to directly control the AI model lifecycle.

Full Package Service​

For customers prioritizing data security, the AI Conductor is installed on the customer’s Private Cloud. By establishing a dedicated cloud, MLOps can be operated within a fortified security framework, ensuring all data and models are securely managed internally.

---

Benefits​

The Report and Solution Services enable partners to create workspaces for individual customers within a single AI Conductor, connecting Edge Conductors and Edge Apps to deliver services. This structure allows partners to efficiently serve multiple customers, with reusable solutions and Edge-based deployment reducing service delivery time.

Customers can minimize the time and cost of infrastructure setup and management, focusing on digital transformation (DX) with the services they need. For customers prioritizing data security, services can be securely operated on their own infrastructure, and by directly managing and operating models, they can assume the role of a partner.

---

Scalability​

By integrating Mellerikat’s cloud infrastructure with Edge Apps, services can be delivered in various forms by deploying Edges across different elements of the business value chain. This enables high scalability across sectors such as product manufacturing, logistics, and customer service.

• Deployment in Diverse Environments: The flexible installation of Edge enables customers to utilize AI solutions in any location, such as factory floors, logistics centers, or offices.
• Expansion Across Business Functions: Edges connected to the cloud enable consistent data management and analysis across the entire business, maximizing overall productivity and efficiency. For example, integrated solutions can address quality control in production lines, logistics optimization, and customer service enhancements.
• Cost Savings: By optimally utilizing resources in a single cloud infrastructure to create and share models, and leveraging local resources at the Edge, costs are reduced. This decreases data transmission costs and optimizes bandwidth usage. Additionally, operations can be sustained without additional infrastructure investments despite expanded service scope.

Mellerikat’s scalability empowers businesses of all sizes to achieve efficient digital transformation, accelerating business innovation.