Sitemap management

Use Sitemap management to control which public Actualog pages search engines can discover, how sitemap XML files are generated, and which complete sitemap is currently published.

This page is available to Application Administrators. Open it from Admin → Services → Sitemap or use Open Sitemap management.

Safe workflow

Use this order whenever you change sitemap configuration:

  1. Change the required settings, cultures, and public content sources.
  2. Select Save settings. The button at the top and the button at the bottom save the same form.
  3. Confirm the saved status beside the lower button.
  4. Select Generate and publish now when you want a new sitemap immediately.
  5. Review Latest generation run and Current published sitemap.
  6. Open the public sitemap index and a sample chunk from Public endpoints.

Generate and publish now uses the last saved settings. Unsaved checkbox or field changes are not included in the run.

Delivery and publication

Public delivery mode

  • Published sitemap serves the last complete sitemap that passed XML validation. This is the recommended mode for normal operation.
  • Live generation builds XML from current database data when a crawler requests an endpoint. It is useful for diagnostics but adds database work to public requests and does not provide the same atomic publication boundary.

A published sitemap is a complete set consisting of sitemap.xml plus every referenced chunk file. Actualog builds and validates the new set separately, then switches the public endpoint only after every file is ready. A crawler therefore never receives a new index that points to incomplete files.

Actualog retains the current publication and one previous publication. The previous copy exists only for safe restoration; technical storage identifiers are intentionally not shown on this page.

Canonical origin

Canonical origin is the scheme and host written into generated crawler URLs, for example https://actualog.com. Enter only the origin: do not include a path, query string, or fragment.

Requests made through a culture host may redirect to the canonical origin before serving XML. This is expected when the saved canonical origin differs from the host used to open the endpoint.

Live fallback

Enable Fall back to live generation when no published sitemap is available to keep sitemap endpoints usable before the first successful publication. Once a valid publication exists, Published sitemap mode serves that validated copy.

Availability and crawler policy

  • Serve sitemap endpoints controls whether sitemap XML endpoints are available. When disabled, sitemap endpoints return HTTP 410.
  • Allow crawling in robots.txt controls whether robots.txt emits Allow: / and advertises the sitemap index. When disabled, it emits Disallow: /.
  • Allow sitemap on staging hosts should normally remain off. Enable it only when a non-production environment is deliberately crawlable.
  • Use culture subdomains for localized URLs produces culture-specific hosts such as en-US.example.com when the deployment supports them.

Disabling sitemap endpoints and disallowing crawling are separate controls. Search engines read robots.txt, while sitemap clients may request the XML endpoints directly.

Generation limits and history

Repository batch size

Repository batch size controls how many source records the generator reads per database batch. The supported range is 100–5,000. A larger value reduces round trips but uses more memory per batch. The default of 1,000 is appropriate for normal environments.

URLs per sitemap file

URLs per sitemap file controls chunk size. The sitemap protocol permits no more than 50,000 URLs per XML file. Actualog creates additional numbered chunk files when a source and culture exceed the selected limit.

Keep generation history

Keep generation history (days) controls automatic retention for completed, failed, and cancelled attempts. The background worker removes expired history every hour.

Clear history immediately hides all visible terminal attempts from the table, regardless of age. Queued or running work is never hidden. The action does not delete published XML files. Hidden terminal records are removed later by the automatic age-based retention sweep, while metadata required by the current publication and the previous restoration point remains protected. New runs appear in the table normally.

Automatic generation

Enable Generate and publish on a schedule to let the background worker create a durable run whenever the current publication becomes due.

Interval (minutes) accepts 15–10,080 minutes. For example:

  • 60 generates hourly;
  • 1440 generates daily;
  • 10080 generates weekly.

Scheduled and manual runs use the same saved configuration, validation, publication, and history workflow. Starting a manual run does not create a second concurrent run when another generation is already queued or running.

Published cultures

Each selected culture generates localized URLs for every enabled public content source.

  • Interface defaults selects active Data Languages that have embedded static RESX resources.
  • English only selects en-US.
  • Select all selects every resource-backed culture shown in the list.
  • The UI badge identifies cultures currently available for interface selection.

At least one culture is required. After changing cultures, select Save settings before generating. Selecting many cultures multiplies the number of source/culture combinations and may increase generation time and file count.

Included public content

Select the public page groups that should appear in generated sitemap files:

  • Categories — public category, catalog, and attribute landing pages.
  • Catalogs — public active catalog pages. A source may contain catalog rows without producing URLs when no row currently satisfies public eligibility rules.
  • Companies — public company profiles.
  • Communities — public community landing pages.
  • Products — products eligible for public search and public product routes.
  • Users — public user profiles.
  • Static pages — supported public marketing and application landing pages.
  • Help topics — localized public Help Content with valid routes.

At least one source is required. Source record counts are diagnostic input counts, not guaranteed URL counts.

Generate and publish now

Select Generate and publish now to queue a durable background run. The request returns immediately; the worker then:

  1. reads the saved settings;
  2. reads eligible public records in batches;
  3. builds the index and chunk XML files;
  4. validates the complete set;
  5. uploads the files to private versioned storage;
  6. switches the public publication pointer only after the complete set succeeds.

If generation fails, the currently published sitemap remains online. Use Retry on a failed run after correcting its cause.

Current publication and restoration

Current published sitemap shows the publication time, URL count, XML file count, culture count, and canonical origin currently served to crawlers.

Select Restore previous publication only when the current sitemap must be replaced with the immediately preceding valid publication. Actualog shows a confirmation dialog before changing the publication pointer. Restoration does not regenerate source data.

Generation history

The history table shows manual, retry, and scheduled attempts with their status, stage, generated URL count, XML file count, culture count, duration, and failure message.

Common states are:

  • Queued — waiting for the background worker;
  • Running — generation or validation is in progress;
  • Completed / Published — the complete sitemap became active;
  • Failed — the run stopped before publication and the previous sitemap stayed active;
  • Cancelled — an administrator stopped an unfinished run.

Use Retry to create a new run with the current saved settings. Retrying does not reuse the failed run's unsaved configuration.

Source data activity

Newest source change (UTC) is the most recent eligible source record update date returned by that repository. It is not the sitemap generation date and it is not proof that the sitemap is stale.

For example, an old Users date means that no eligible public user record has changed since that date. Compare sitemap freshness with Current published sitemap → Published and Latest generation run, not with source change dates alone.

Public endpoints

  • Sitemap index is the crawler entry point and lists every generated culture/source chunk.
  • Sample chunk sitemap verifies one generated chunk route.
  • robots.txt shows the effective crawler directives and sitemap index URL.

Published endpoints return an ETag and Last-Modified value. These headers let crawlers avoid downloading unchanged XML repeatedly.

Troubleshooting

My changes were not used by generation

Confirm that Save settings completed before you selected Generate and publish now. The generator reads persisted SettingsStore values, not unsaved browser fields.

Clear history did not remove queued or running rows

This is intentional. Active work is protected. Cancel an unwanted active run first, wait until it becomes terminal, and then clear history.

Generation produced zero URLs

Check that at least one culture and one public content source are selected, the canonical origin is valid, and the selected repositories contain records eligible for public routes. A non-zero source count does not guarantee that every record is publicly eligible.

Sitemap endpoints return HTTP 410

Enable Serve sitemap endpoints, save settings, and confirm that the current environment is allowed. On staging, also review Allow sitemap on staging hosts.

robots.txt disallows crawling

Enable Allow crawling in robots.txt and save settings. This setting changes crawler directives but does not by itself enable disabled sitemap endpoints.

Restore previous publication is unavailable

A restore action requires two successful retained publications. Generate and publish another valid sitemap before expecting a previous restoration point.

A run failed

Read its failure message, correct the corresponding setting or data condition, save settings, and select Retry. The current published sitemap remains available during the repair.

Use Open Help Management to maintain Help Content included by the Help topics source.