Running Your First Agentic Workflow with a Local LLM and n8n

I’ve built a flow using local n8n and a local LLM. Below is a quick-start guide so you can reproduce (and improve on) what I built.

What we’ll cover

A quick intro to agentic workflows
How to set up a local LLM with Ollama
How to run n8n locally via Docker
A simple use case: receive a PDF, extract its text, send that text to your LLM, parse the response, and return only the name to whom it’s addressed

First, what are an agent, agentic AI, and agentic automation?

Term	Meaning
Agent	An entity that can perceive, decide, and act.
Agentic AI	An autonomous system capable of learning and adapting based on its environment.
Agentic automation	The combination of agentic AI with automation, enabling systems to carry out complex, adaptive tasks without constant human oversight.

These concepts are at the forefront of AI innovation, paving the way for smarter, more capable systems that are transforming industries and changing the way we work.

Setting Up Your Local LLM with Ollama

What’s an LLM?

A Large Language Model is a very big neural network trained on massive text corpora so it can generate, understand, and reason over natural language (think ChatGPT, Llama 3, Gemini, etc.).

What’s Ollama?

Ollama isn’t itself an LLM; it’s an open-source desktop/CLI platform that lets you download, run, and swap many LLMs locally on macOS, Linux, or Windows with one-line commands. Think of it as a lightweight “package-manager + runtime” for models such as Llama 3, Gemma 2b, DeepSeek-R1, and new multimodal releases.

Why use Ollama?

Data privacy: Everything stays on your machine.
Cost efficiency: No ongoing cloud fees.
Customization: Fine-tune or modify models for your specific tasks.
Offline capability: Once set up, it runs without an internet connection.
Self-hosted flexibility: Perfect for integrating into custom applications or systems.

I first tried phi (a small, lightweight model) but found its accuracy on noisy PDFs lacking, so I switched to gemma:2b, which strikes a good balance between size and reasoning power.

Installing Ollama (Windows)

Download Ollama from https://ollama.com and install it
Run a model

ollama run gemma:2b

3. Test with a quick prompt

curl http://localhost:11434/api/generate \
-d '{"model":"gemma:2b","prompt":"What is the capital of Turkey?"}'

If you receive a valid response, your LLM is ready.

Running n8n Locally with Docker

Why Docker?

Isolated environments: Avoid dependency conflicts.
Clean system: Keep your OS free of global installs.
Quick setup: Spin up repeatable environments with a single command.

Step-by-step

Install Docker Desktop: https://www.docker.com/products/docker-desktop
In a project folder, create docker-compose.yml:

version: "3.1"
services:
n8n:
image: n8nio/n8n
ports:
- "5678:5678"
volumes:
- n8n_data:/home/node/.n8n
environment:
- N8N_BASIC_AUTH_ACTIVE=true
- N8N_BASIC_AUTH_USER=admin
- N8N_BASIC_AUTH_PASSWORD=admin
- TZ=Europe/Istanbul
volumes:
n8n_data:

3. Start n8n:

docker-compose up -d

4. Visit http://localhost:5678 and log in with the credentials above.

Building the Workflow

Below is the finished flow you’ll create:

[Receive PDF (Webhook)] → [Extract from PDF] → [HTTP Request to Gemma2b] → [Code]

Webhook (Receive PDF)

Drag in a Webhook node.
Method: POST.
Path: upload-pdf
Save and copy the test URL (you’ll need it to post a sample PDF later).

2. Extract from PDF

Click the little “+” that appears on the Webhook node’s right edge and add “Extract from PDF” (community node) or the “PDF Extract” built-in node if you have it.
Binary Property: leave as data (the default).
Text Property Name: set to text (this will give us {{$json.text}} downstream).

3. Ask Gemma2b (HTTP Request)

Add an HTTP Request node.
Method: POST
URL: http://host.docker.internal:11434/api/generate
Body Content Type: JSON → Raw (Disabled)
JSON/RAW Parameters:

{
"model": "gemma:2b",
"prompt": "From the following text: {{ $json.text }}, please extract the full name of the person to whom the document is addressed.
Format your response exactly as:
Full Name: [Name]"
}

Code (Parse the response)

Add a Code node (JavaScript).
Paste:

const raw = $json["data"];
const lines = raw.split("\n").filter(l => l.trim() !== "");

let fullResponse = "";
for (const line of lines) {
  const parsed = JSON.parse(line);
  fullResponse += parsed.response;
}

return [{ json: { result: fullResponse } }];

This converts the LLM’s raw string into a clean JSON field.

4. Activate & Test.

You’ve just built a fully local, privacy-friendly agentic workflow!

LLM Prompt

System prompt
From the following text: {{ $json.text }}, please extract the full name of the person to whom the document is addressed.
Format your response exactly as:
Full Name: [Name]

We’ll explore more agentic use cases in future posts.
If you’ve built something similar, I’d love to hear your approach.

See you next time! 🚀✨

Kardelen Cihangir