Inferensys

Glossary

Semantic Kernel

Semantic Kernel is an open-source SDK from Microsoft that enables the integration of conventional programming languages with large language models, facilitating the creation of planners and semantic functions that can call native code.
Stylish home-office setup in a modern highrise apartment, floor-to-ceiling windows showing city skyline at golden hour, a laptop displaying a beautiful semantic search interface.
FUNCTION CALLING FRAMEWORK

What is Semantic Kernel?

Semantic Kernel is an open-source software development kit from Microsoft that enables the integration of conventional programming languages with large language models, facilitating the creation of planners and semantic functions that can call native code.

Semantic Kernel is an open-source SDK from Microsoft that enables developers to integrate Large Language Models (LLMs) with conventional programming languages like C# and Python. It provides a framework for creating semantic functions (prompt-based, AI-powered routines) and native functions (traditional code), allowing them to be orchestrated together by a planner to execute complex, multi-step tasks. This architecture is central to building AI agents capable of tool calling and interacting with external APIs and systems.

The SDK abstracts the complexities of prompt engineering and function calling, offering a planner component that can dynamically generate and execute sequences of actions. It supports plugins for modular capability extension and integrates with vector databases for Retrieval-Augmented Generation (RAG). By bridging symbolic AI (code) with subsymbolic AI (LLMs), Semantic Kernel provides a structured foundation for developing reliable, production-grade agentic applications that leverage both deterministic logic and generative reasoning.

FUNCTION CALLING FRAMEWORKS

Core Components of Semantic Kernel

Semantic Kernel is an open-source SDK that integrates conventional programming languages with large language models, enabling the creation of planners and semantic functions that can call native code. Its architecture is built around several key abstractions.

01

Plugins

Plugins are the primary unit of capability extension in Semantic Kernel. They are collections of related functions that an AI can call. A plugin can contain:

  • Native Functions: Code written in languages like C#, Python, or Java that performs deterministic operations (e.g., calculations, database queries).
  • Semantic Functions: Prompts defined by natural language instructions and few-shot examples, executed by an LLM. Plugins modularize skills—like "MathPlugin" or "EmailPlugin"—and are registered with the kernel to make their functions discoverable and callable by planners.
02

Kernel

The Kernel is the central runtime and orchestration engine. It acts as a function registry and dispatcher, managing:

  • Plugin Registration: Loading and cataloging available plugins and their functions.
  • Function Invocation: Executing both native and semantic functions, handling input/output binding.
  • Context Management: Maintaining a shared KernelArguments context that flows between chained function calls, allowing results from one function to be passed as input to another. The kernel abstracts the complexity of mixing LLM prompts with traditional code execution.
03

Planners

Planners are AI-driven components that automatically decompose a high-level user goal into a sequence of executable steps (a plan). Using the kernel's registry, a planner:

  1. Analyzes the available plugins and functions.
  2. Generates a step-by-step plan to achieve the goal.
  3. Invokes the kernel to execute each step in the plan. For example, given the goal "Summarize the top news story and email it to me," a planner might create a plan calling a NewsPlugin.GetTopStory function followed by an EmailPlugin.Send function.
04

Semantic Functions

Semantic functions are prompts treated as callable software functions. Defined by:

  • Prompt Template: A natural language instruction with placeholders for variables (e.g., Summarize this: {{$input}}).
  • Configuration: Settings like the LLM model to use, temperature, and token limits. When invoked, the kernel renders the template with provided arguments, sends it to the configured LLM, and returns the completion. This allows developers to package and reuse complex prompt logic alongside native code.
05

Connectors

Connectors are abstractions for interfacing with external AI and data services. Key connector types include:

  • AI Service Connectors: Provide a uniform interface to different LLM endpoints (e.g., OpenAI, Azure OpenAI, Hugging Face). Switching models often requires only a configuration change.
  • Memory Connectors: Interface with vector databases (e.g., Azure AI Search, Qdrant, PostgreSQL) to store and retrieve embeddings for semantic memory and recall.
  • Chat Completion Connectors: Specialized connectors for managing multi-turn conversational context with chat models. Connectors standardize integration, promoting portability and reducing vendor lock-in.
06

Memories

The Memory abstraction provides AI agents with persistent, searchable context beyond a single conversation. It enables:

  • Vector-based Semantic Search: Storing text as embeddings in a vector database allows the kernel to retrieve relevant past information using similarity search, not just keyword matching.
  • Information Recall: A planner can query memory to find relevant facts before executing a step (e.g., "Find the user's project notes from last week").
  • Chat History: Maintaining long-term conversation context across sessions. This component is crucial for building agents that learn and act on historical data.
FUNCTION CALLING FRAMEWORK

How Semantic Kernel Works

Semantic Kernel is an open-source orchestration SDK from Microsoft that integrates conventional programming languages with large language models to build AI agents capable of executing native code.

Semantic Kernel is an open-source software development kit (SDK) that enables developers to create AI agents by seamlessly blending traditional programming with large language model (LLM) capabilities. It provides a planner that can decompose complex goals into executable steps, and a kernel that orchestrates the invocation of both semantic functions (prompt-based LLM calls) and native plugin functions (pre-written code). This architecture allows an agent to dynamically reason and act using a unified set of tools.

At its core, the framework uses a function registry where all capabilities—whether AI-driven prompts or conventional APIs—are registered as callable plugins. When an agent receives a request, its planner can generate a sequence of these functions. The kernel's dynamic dispatch system then executes them, handling parameter validation, structured output parsing, and error propagation. This enables the construction of reliable, multi-step workflows where LLMs orchestrate calls to external systems and proprietary business logic.

SEMANTIC KERNEL

Frequently Asked Questions

Essential questions about Microsoft's Semantic Kernel, an open-source SDK for integrating Large Language Models with conventional programming to build AI agents and planners.

Semantic Kernel is an open-source software development kit (SDK) from Microsoft that enables developers to integrate conventional programming languages (like C# and Python) with large language models (LLMs) to create AI agents and planners. It works by providing a planner that can decompose high-level goals into a sequence of executable steps, and a kernel that orchestrates the execution of both semantic functions (LLM-powered prompts) and native functions (traditional code). Developers define skills as collections of functions, which the kernel dynamically discovers and chains together based on natural language instructions, allowing the AI to reason and act using both generative AI and deterministic code.

Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.