Quick start: Azure with Azure Monitor Agent
Use this when your Linux servers already run Azure Monitor Agent (AMA): Azure VMs, scale sets,
or Azure Arc-enabled servers anywhere. Bower runs on each server, selects and redacts the
security events, then hands them to the local syslog daemon. AMA's syslog collection carries
them to the same BowerSecurity_CL table in Sentinel. Bower needs no Azure credential on the
server; AMA's managed identity does the upload.
Settings, sizes, failure codes and limits are in the syslog output reference. This page is the steps.
Moving a server from azure-logs-ingestion or ama-spool? Do the steps in order. Bower
counts an event delivered as soon as the local daemon takes it, so if you switch the output
before the rule is associated, those events are removed from Bower's queue and never uploaded.
Before you choose this path
- Any local account on the server can write rows into
BowerSecurity_CLon this path, past Bower's policy and redaction, unless you install the origin filter in step 3 (then root and theboweraccount still can). Where local accounts are not trusted, useBOWER_OUTPUT=azure-logs-ingestion(Azure ingestion API). - Bower cannot confirm arrival here: only a Sentinel query can. Events larger than 5,730 bytes of JSON are refused. Compare the three Azure paths in Which Azure path.
- Windows servers: use
azure-logs-ingestion; this output is refused on Windows.
Before you begin
- The Bower Sentinel resources from
deploy/bicep/main.bicep, which create theBowerSecurity_CLtable. See Azure ingestion API. - AMA installed on the server.
- A free syslog facility for Bower. The default is
local4; pick another oflocal0tolocal7if something on the server already uses it. - Bower installed from the deb or rpm package, with its output left as it is for now.
1. Create the collection rule
On an admin workstation with the Azure CLI and a checkout of this repository:
# Preview first; nothing is created until you run "create".
az deployment group what-if --resource-group rg-sentinel \
--template-file deploy/bicep/ama-syslog-dcr.bicep \
--parameters workspaceName=law-sentinel
az deployment group create --resource-group rg-sentinel --name bower-ama-syslog \
--template-file deploy/bicep/ama-syslog-dcr.bicep \
--parameters workspaceName=law-sentinel
Add workspaceResourceGroup=... if the workspace is in another resource group, and
facility=local5 (for example) if you will not use local4. The rule must be in the
workspace's region: add location=<workspace region> when the resource group is in another
region. For Private Link, see
the collection rule first: each machine
also needs its own endpoint association.
2. Associate the rule with each server
On the admin workstation:
RULE_ID=$(az deployment group show --resource-group rg-sentinel --name bower-ama-syslog \
--query properties.outputs.dataCollectionRuleId.value -o tsv)
az monitor data-collection rule association create \
--name bower-ama-syslog \
--rule-id "$RULE_ID" \
--resource <vm-or-arc-machine-resource-id>
bower-ama-syslog in az deployment group show is the deployment name from step 1. The
association name is per machine, so the same name works on every server. For many servers, assign the built-in Azure
Policy definition Configure Linux Machines to be associated with a Data Collection Rule or a
Data Collection Endpoint
(Microsoft), with this
rule's ID.
AMA then writes its rsyslog forwarding file and restarts rsyslog (Microsoft).
3. Prepare the server
On the server:
systemctl is-active rsyslog # must print "active"
ls /etc/rsyslog.d/ | grep azuremonitoragent # the agent's forwarding file
If rsyslog is not installed, install it (sudo dnf install rsyslog or sudo apt install rsyslog) and enable it, then wait for the agent to write its file. Microsoft documents the file
as 10-azuremonitoragent-omfwd.conf, and that is the name agent 1.45.1 wrote on Oracle Linux
9.8 and Ubuntu 24.04 in the live run (#249). It appears a few minutes after the association.
Do not go on until it exists.
Then raise the limits that drop bursts and, if you want them, install the filters. Copy the
files from
examples/cloud-agents/azure;
each file says where it goes and how it was tested.
| File | Where | Why |
|---|---|---|
bower-ama-syslog-ratelimit.conf | /etc/systemd/system/bower.service.d/ama-syslog-output.conf | journald's limit for bower.service only |
rsyslog-imjournal-ratelimit.conf | edit /etc/rsyslog.conf (RHEL, Rocky, AlmaLinux) | rsyslog's host-wide limit of about 33 events per second |
sysctl-bower-syslog.conf | /etc/sysctl.d/60-bower-syslog.conf (Debian, Ubuntu) | journald's forward to rsyslog drops messages in a burst |
rsyslog-bower-origin-imjournal.conf or rsyslog-bower-origin-imuxsock.conf | /etc/rsyslog.d/09-bower-origin.conf | Only bower.service can write rows (why) |
rsyslog-bower-local-copies.conf | /etc/rsyslog.d/11-bower-local-copies.conf | Keeps Bower events out of /var/log/messages and /var/log/syslog (why) |
sudo rsyslogd -N1 # the configuration is valid
sudo systemctl daemon-reload
sudo systemctl restart rsyslog
4. Switch Bower's output
On the server, in /etc/bower/bower.env, uncomment the syslog block (or add these lines):
BOWER_OUTPUT=ama-syslog
#BOWER_AMA_SYSLOG_FACILITY=local4 # match the rule's facility parameter
#BOWER_AMA_SYSLOG_MAX_BYTES=7680 # at least 512 below rsyslog's MaxMessageSize
sudo systemctl restart bower
bower doctor --syslog-socket /dev/log
5. Send a test event and check it
On the server:
bower test emit
It sends a new synthetic authentication failure each time and prints the collector's answer,
including "eventId".
On a fresh install the collector answers 403 ("Producer is not authorized for this event
identity"): producer scopes are default deny, and nothing is allowed to claim the canary's
identity yet. For the test, allow it for the loopback producer in a scopes file, then restart
Bower (see Producer identity scopes):
sudo tee /etc/bower/producer-scopes.json >/dev/null <<JSON
[{"producer": "anonymous",
"application": {"name": "BowerCanary", "environment": "test", "instance": "$(cat /proc/sys/kernel/hostname)"},
"source": {"ipAddress": "192.0.2.1"}}]
JSON
sudo chown root:bower /etc/bower/producer-scopes.json && sudo chmod 0640 /etc/bower/producer-scopes.json
echo 'BOWER_PRODUCER_SCOPES_FILE=/etc/bower/producer-scopes.json' | sudo tee -a /etc/bower/bower.env
sudo systemctl restart bower
Use "producer": "default" instead when the collector has a shared ingest token. Add your
applications' identities to the same file. If the collector has an ingest token, pass it from
the file without printing it:
sudo sh -c 'BOWER_INGEST_TOKEN="$(cat /etc/bower/secrets/ingest-token)" bower test emit'
Check that it reached the local daemon:
journalctl -t bower -n 1 -o cat
In Sentinel, after a few minutes, with the ID it printed:
BowerSecurity_CL
| where EventId == "<eventId from bower test emit>"
If the row does not arrive, split the path in half:
- Not in
journalctl -t bower: check Bower's status,curl -s http://127.0.0.1:4319/v1/status(with-H "Authorization: Bearer ..."if a token is set) and itsdeliveryActivity.lastFailureCode. - In the journal but not in Sentinel: check that the agent's file is in
/etc/rsyslog.d/, that no earlier rsyslog rule discards the facility, the rule association, and/var/opt/microsoft/azuremonitoragent/log/mdsd.err.
Remove
-
Server: set
BOWER_OUTPUTto another output (ornone) and restart Bower first, so no events are handed to a daemon that no longer forwards them. Remove the files from step 3 if you added them. -
Admin workstation: delete the association on each server, then the rule (named
<namePrefix>-ama-syslog-dcr):az monitor data-collection rule association delete --name bower-ama-syslog \--resource <vm-or-arc-machine-resource-id>az monitor data-collection rule delete --resource-group rg-sentinel \--name bower-ama-syslog-dcr
Rows already in BowerSecurity_CL stay until the table's retention removes them.
Limits
- Linux with rsyslog only. See the reference.
- This path was run once end to end with real Azure Monitor Agents (1.45.1) on Oracle Linux 9.8 and Ubuntu 24.04 (#249): test events, 5,000-event bursts and the shipped filters. The results are in the reference. It was also run once behind Private Link. Arc machines, syslog-ng and other agent versions were not run.