iommu/amd: Fix ineffective error check in nested domain allocation

amd_iommu_pdom_id_alloc() returns an int: a domain ID on success, or the
negative errno from ida_alloc_range() when the ID space is exhausted or
memory is short.  amd_iommu_alloc_domain_nested() stores that return value
in gdom_info->hdom_id, which is a u32, and only then tests it:

	gdom_info->hdom_id = amd_iommu_pdom_id_alloc();
	if (gdom_info->hdom_id <= 0) {

The assignment discards the sign, so -ENOSPC becomes 0xffffffe4 and the
test never fires.  The nested domain is then set up with a host domain ID
that was never allocated, instead of the allocation failing with -ENOSPC.

Keep the value in an int, test it there, and store it only once it is
known to be valid, which is what the other amd_iommu_pdom_id_alloc()
callers already do.

Fixes: 757d2b1fdf ("iommu/amd: Introduce gDomID-to-hDomID Mapping and handle parent domain invalidation")
Signed-off-by: Hemanth Selam <hemanth.selam@gmail.com>
Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
This commit is contained in:
Hemanth Selam 2026-08-25 15:35:54 +05:30 • committed by Joerg Roedel
commit fa5c0827f0