CUDA Driver API (PDF) - v11.0.207 (older) - Last updated June 29, 2020 - Send Feedback

## 5.12. Virtual Memory Management

This section describes the virtual memory management functions of the low-level CUDA driver application programming interface.

### Functions

CUresult cuMemAddressFree ( CUdeviceptr ptr, size_t size )
  Free an address range reservation.
CUresult cuMemAddressReserve ( CUdeviceptr* ptr, size_t size, size_t alignment, CUdeviceptr addr, unsigned long long flags )
  Allocate an address range reservation.
CUresult cuMemCreate ( CUmemGenericAllocationHandle* handle, size_t size, const CUmemAllocationProp* prop, unsigned long long flags )
  Create a shareable memory handle representing a memory allocation of a given size described by the given properties.
CUresult cuMemExportToShareableHandle ( void* shareableHandle, CUmemGenericAllocationHandle handle, CUmemAllocationHandleType handleType, unsigned long long flags )
  Exports an allocation to a requested shareable handle type.
CUresult cuMemGetAccess ( unsigned long long* flags, const CUmemLocation* location, CUdeviceptr ptr )
  Get the access `flags` set for the given `location` and `ptr`.
CUresult cuMemGetAllocationGranularity ( size_t* granularity, const CUmemAllocationProp* prop, CUmemAllocationGranularity_flags option )
  Calculates either the minimal or recommended granularity.
CUresult cuMemGetAllocationPropertiesFromHandle ( CUmemAllocationProp* prop, CUmemGenericAllocationHandle handle )
  Retrieve the contents of the property structure defining properties for this handle.
CUresult cuMemImportFromShareableHandle ( CUmemGenericAllocationHandle* handle, void* osHandle, CUmemAllocationHandleType shHandleType )
  Imports an allocation from a requested shareable handle type.
CUresult cuMemMap ( CUdeviceptr ptr, size_t size, size_t offset, CUmemGenericAllocationHandle handle, unsigned long long flags )
  Maps an allocation handle to a reserved virtual address range.
CUresult cuMemRelease ( CUmemGenericAllocationHandle handle )
  Release a memory handle representing a memory allocation which was previously allocated through cuMemCreate.
CUresult cuMemRetainAllocationHandle ( CUmemGenericAllocationHandle* handle, void* addr )
  Given an address `addr`, returns the allocation handle of the backing memory allocation.
CUresult cuMemSetAccess ( CUdeviceptr ptr, size_t size, const CUmemAccessDesc* desc, size_t count )
  Set the access flags for each location specified in `desc` for the given virtual address range.
CUresult cuMemUnmap ( CUdeviceptr ptr, size_t size )
  Unmap the backing memory of a given address range.

### Functions

CUresult cuMemAddressFree ( CUdeviceptr ptr, size_t size )
  Free an address range reservation.

  ###### Parameters

  `ptr`
    - Starting address of the virtual address range to free
  `size`
    - Size of the virtual address region to free

  ###### Returns

  CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED, CUDA_ERROR_NOT_SUPPORTED

  ###### Description

  Frees a virtual address range reserved by cuMemAddressReserve. The size must match what was given to memAddressReserve and the ptr given must match what was returned from memAddressReserve.

  See also:

  cuMemAddressReserve

CUresult cuMemAddressReserve ( CUdeviceptr* ptr, size_t size, size_t alignment, CUdeviceptr addr, unsigned long long flags )
  Allocate an address range reservation.

  ###### Parameters

  `ptr`
    - Resulting pointer to start of virtual address range allocated
  `size`
    - Size of the reserved virtual address range requested
  `alignment`
    - Alignment of the reserved virtual address range requested
  `addr`
    - Fixed starting address range requested
  `flags`
    - Currently unused, must be zero

  ###### Returns

  CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_OUT_OF_MEMORY, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED, CUDA_ERROR_NOT_SUPPORTED

  ###### Description

  Reserves a virtual address range based on the given parameters, giving the starting address of the range in `ptr`. This API requires a system that supports UVA. The size and address parameters must be a multiple of the host page size and the alignment must be a power of two or zero for default alignment.

  See also:

  cuMemAddressFree

CUresult cuMemCreate ( CUmemGenericAllocationHandle* handle, size_t size, const CUmemAllocationProp* prop, unsigned long long flags )
  Create a shareable memory handle representing a memory allocation of a given size described by the given properties.

  ###### Parameters

  `handle`
    - Value of handle returned. All operations on this allocation are to be performed using this handle.
  `size`
    - Size of the allocation requested
  `prop`
    - Properties of the allocation to create.
  `flags`
    - flags for future use, must be zero now.

  ###### Returns

  CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_OUT_OF_MEMORY, CUDA_ERROR_INVALID_DEVICE, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED, CUDA_ERROR_NOT_SUPPORTED

  ###### Description

  This creates a memory allocation on the target device specified through the `prop` strcuture. The created allocation will not have any device or host mappings. The generic memory `handle` for the allocation can be mapped to the address space of calling process via cuMemMap. This handle cannot be transmitted directly to other processes (see cuMemExportToShareableHandle). On Windows, the caller must also pass an LPSECURITYATTRIBUTE in `prop` to be associated with this handle which limits or allows access to this handle for a recepient process (see CUmemAllocationProp::win32HandleMetaData for more). The `size` of this allocation must be a multiple of the the value given via cuMemGetAllocationGranularity with the CU_MEM_ALLOC_GRANULARITY_MINIMUM flag.

  Note:

  Note that this function may also return error codes from previous, asynchronous launches.

  See also:

  cuMemRelease, cuMemExportToShareableHandle, cuMemImportFromShareableHandle

CUresult cuMemExportToShareableHandle ( void* shareableHandle, CUmemGenericAllocationHandle handle, CUmemAllocationHandleType handleType, unsigned long long flags )
  Exports an allocation to a requested shareable handle type.

  ###### Parameters

  `shareableHandle`
    - Pointer to the location in which to store the requested handle type
  `handle`
    - CUDA handle for the memory allocation
  `handleType`
    - Type of shareable handle requested (defines type and size of the `shareableHandle` output parameter)
  `flags`
    - Reserved, must be zero

  ###### Returns

  CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED, CUDA_ERROR_NOT_SUPPORTED

  ###### Description

  Given a CUDA memory handle, create a shareable memory allocation handle that can be used to share the memory with other processes. The recipient process can convert the shareable handle back into a CUDA memory handle using cuMemImportFromShareableHandle and map it with cuMemMap. The implementation of what this handle is and how it can be transferred is defined by the requested handle type in `handleType`

  Once all shareable handles are closed and the allocation is released, the allocated memory referenced will be released back to the OS and uses of the CUDA handle afterward will lead to undefined behavior.

  This API can also be used in conjunction with other APIs (e.g. Vulkan, OpenGL) that support importing memory from the shareable type

  See also:

  cuMemImportFromShareableHandle

CUresult cuMemGetAccess ( unsigned long long* flags, const CUmemLocation* location, CUdeviceptr ptr )
  Get the access `flags` set for the given `location` and `ptr`.

  ###### Parameters

  `flags`
    - Flags set for this location
  `location`
    - Location in which to check the flags for
  `ptr`
    - Address in which to check the access flags for

  ###### Returns

  CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_DEVICE, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED, CUDA_ERROR_NOT_SUPPORTED

  ###### Description

  See also:

  cuMemSetAccess

CUresult cuMemGetAllocationGranularity ( size_t* granularity, const CUmemAllocationProp* prop, CUmemAllocationGranularity_flags option )
  Calculates either the minimal or recommended granularity.

  ###### Parameters

  `granularity`
    Returned granularity.
  `prop`
    Property for which to determine the granularity for
  `option`
    Determines which granularity to return

  ###### Returns

  CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED, CUDA_ERROR_NOT_SUPPORTED

  ###### Description

  Calculates either the minimal or recommended granularity for a given allocation specification and returns it in granularity. This granularity can be used as a multiple for alignment, size, or address mapping.

  See also:

  cuMemCreate, cuMemMap

CUresult cuMemGetAllocationPropertiesFromHandle ( CUmemAllocationProp* prop, CUmemGenericAllocationHandle handle )
  Retrieve the contents of the property structure defining properties for this handle.

  ###### Parameters

  `prop`
    - Pointer to a properties structure which will hold the information about this handle
  `handle`
    - Handle which to perform the query on

  ###### Returns

  CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED, CUDA_ERROR_NOT_SUPPORTED

  ###### Description

  See also:

  cuMemCreate, cuMemImportFromShareableHandle

CUresult cuMemImportFromShareableHandle ( CUmemGenericAllocationHandle* handle, void* osHandle, CUmemAllocationHandleType shHandleType )
  Imports an allocation from a requested shareable handle type.

  ###### Parameters

  `handle`
    - CUDA Memory handle for the memory allocation.
  `osHandle`
    - Shareable Handle representing the memory allocation that is to be imported.
  `shHandleType`
    - handle type of the exported handle CUmemAllocationHandleType.

  ###### Returns

  CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED, CUDA_ERROR_NOT_SUPPORTED

  ###### Description

  If the current process cannot support the memory described by this shareable handle, this API will error as CUDA_ERROR_NOT_SUPPORTED.

  Note:

  Importing shareable handles exported from some graphics APIs(Vulkan, OpenGL, etc) created on devices under an SLI group may not be supported, and thus this API will return CUDA_ERROR_NOT_SUPPORTED. There is no guarantee that the contents of `handle` will be the same CUDA memory handle for the same given OS shareable handle, or the same underlying allocation.

  See also:

  cuMemExportToShareableHandle, cuMemMap, cuMemRelease

CUresult cuMemMap ( CUdeviceptr ptr, size_t size, size_t offset, CUmemGenericAllocationHandle handle, unsigned long long flags )
  Maps an allocation handle to a reserved virtual address range.

  ###### Parameters

  `ptr`
    - Address where memory will be mapped.
  `size`
    - Size of the memory mapping.
  `offset`
    - Offset into the memory represented by
    - `handle` from which to start mapping
    - Note: currently must be zero.
  `handle`
    - Handle to a shareable memory
  `flags`
    - flags for future use, must be zero now.

  ###### Returns

  CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_DEVICE, CUDA_ERROR_OUT_OF_MEMORY, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED, CUDA_ERROR_NOT_SUPPORTED

  ###### Description

  Maps bytes of memory represented by `handle` starting from byte `offset` to `size` to address range [`addr`, `addr` + `size`]. This range must be an address reservation previously reserved with cuMemAddressReserve, and `offset` + `size` must be less than the size of the memory allocation. Both `ptr`, `size`, and `offset` must be a multiple of the value given via cuMemGetAllocationGranularity with the CU_MEM_ALLOC_GRANULARITY_MINIMUM flag.

  Please note calling cuMemMap does not make the address accessible, the caller needs to update accessibility of a contiguous mapped VA range by calling cuMemSetAccess.

  Once a recipient process obtains a shareable memory handle from cuMemImportFromShareableHandle, the process must use cuMemMap to map the memory into its address ranges before setting accessibility with cuMemSetAccess.

  cuMemMap can only create mappings on VA range reservations that are not currently mapped.

  Note:

  Note that this function may also return error codes from previous, asynchronous launches.

  See also:

  cuMemUnmap, cuMemSetAccess, cuMemCreate, cuMemAddressReserve, cuMemImportFromShareableHandle

CUresult cuMemRelease ( CUmemGenericAllocationHandle handle )
  Release a memory handle representing a memory allocation which was previously allocated through cuMemCreate.

  ###### Parameters

  `handle`
    Value of handle which was returned previously by cuMemCreate.

  ###### Returns

  CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED, CUDA_ERROR_NOT_SUPPORTED

  ###### Description

  Frees the memory that was allocated on a device through cuMemCreate.

  The memory allocation will be freed when all outstanding mappings to the memory are unmapped and when all outstanding references to the handle (including it's shareable counterparts) are also released. The generic memory handle can be freed when there are still outstanding mappings made with this handle. Each time a recepient process imports a shareable handle, it needs to pair it with cuMemRelease for the handle to be freed. If `handle` is not a valid handle the behavior is undefined.

  Note:

  Note that this function may also return error codes from previous, asynchronous launches.

  See also:

  cuMemCreate

CUresult cuMemRetainAllocationHandle ( CUmemGenericAllocationHandle* handle, void* addr )
  Given an address `addr`, returns the allocation handle of the backing memory allocation.

  ###### Parameters

  `handle`
    CUDA Memory handle for the backing memory allocation.
  `addr`
    Memory address to query, that has been mapped previously.

  ###### Returns

  CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED, CUDA_ERROR_NOT_SUPPORTED

  ###### Description

  The handle is guaranteed to be the same handle value used to map the memory. If the address requested is not mapped, the function will fail. The returned handle must be released with corresponding number of calls to cuMemRelease.

  Note:

  The address `addr`, can be any address in a range previously mapped by cuMemMap, and not necessarily the start address.

  See also:

  cuMemCreate, cuMemRelease, cuMemMap

CUresult cuMemSetAccess ( CUdeviceptr ptr, size_t size, const CUmemAccessDesc* desc, size_t count )
  Set the access flags for each location specified in `desc` for the given virtual address range.

  ###### Parameters

  `ptr`
    - Starting address for the virtual address range
  `size`
    - Length of the virtual address range
  `desc`
    - Array of CUmemAccessDesc that describe how to change the
    - mapping for each location specified
  `count`
    - Number of CUmemAccessDesc in `desc`

  ###### Returns

  CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_DEVICE, CUDA_ERROR_NOT_SUPPORTED

  ###### Description

  Given the virtual address range via `ptr` and `size`, and the locations in the array given by `desc` and `count`, set the access flags for the target locations. The range must be a fully mapped address range containing all allocations created by cuMemMap / cuMemCreate.

  Note:

  - Note that this function may also return error codes from previous, asynchronous launches.

  - This function exhibits synchronous behavior for most use cases.

  See also:

  cuMemSetAccess, cuMemCreate, :cuMemMap

CUresult cuMemUnmap ( CUdeviceptr ptr, size_t size )
  Unmap the backing memory of a given address range.

  ###### Parameters

  `ptr`
    - Starting address for the virtual address range to unmap
  `size`
    - Size of the virtual address range to unmap

  ###### Returns

  CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED, CUDA_ERROR_NOT_SUPPORTED

  ###### Description

  The range must be the entire contiguous address range that was mapped to. In other words, cuMemUnmap cannot unmap a sub-range of an address range mapped by cuMemCreate / cuMemMap. Any backing memory allocations will be freed if there are no existing mappings and there are no unreleased memory handles.

  When cuMemUnmap returns successfully the address range is converted to an address reservation and can be used for a future calls to cuMemMap. Any new mapping to this virtual address will need to have access granted through cuMemSetAccess, as all mappings start with no accessibility setup.

  Note:

  - Note that this function may also return error codes from previous, asynchronous launches.

  - This function exhibits synchronous behavior for most use cases.

  See also:

  cuMemCreate, cuMemAddressReserve