java.lang.Error: Invalid memory access on 32 bit Windows

I have JNA wrapper for a C DLL. It works fine, except when used on a Windows 32-bit system. Here is a simplified example:

int SetData(const wchar_t* data);

int SetId(const wchar_t* id, uint32_t flags);

I created JNA bindings as follows:

public static native int SetData(WString data);

public static native int SetId(WString id, int flags);

The first function SetData() works fine on both 32-bit as well as 64-bit Windows, but the second function crashes on Windows 7 32-bit.

I tried using NativeLong as suggested in other related posts, but it didn’t help.

Here is the link to the repository:

https://github.com/cryptlex/lexactivator-java/blob/master/src/main/java/com/cryptlex/lexactivator/LexActivatorNative.java

Answer

Your mappings are correct: WString is the correct mapping for const wchar_t*, and int (always 32 bits in Java) is the correct mapping for uint32_t (always 32 bits), with a caveat about signededness that shouldn’t matter when used as a flags bitmask.

I’m not sure where you read that NativeLong would be appropriate here. This is primarily intended for *nix native code in which sizeof(long) differs based on the OS bitness. Practically, it doesn’t actually matter on Windows since LONG is always 32-bit, but it involves unnecessary object creation vs. a primitive.

The “Invalid Memory Access” error thrown by JNA is a “graceful” handling of native memory errors caught by Structured Exception Handling. All you really need to know is that either You are attempting to access native memory that you do not own or Your native memory allocation failed.

Debugging these errors always involves carefully tracking memory allocations. When is memory allocated? When is it released? Who (Java-side or native side) is responsible for this allocation? Your program is likely crashing at the point you attempt to copy data from the user-provided ID string to some native memory that your native DLL is accessing. So that points to two memory pointers you need to investigate.

• Look at the memory where the ID string is being written. Find out when the memory for it is allocated. Ensure it is large enough for the string (should be 2x string length + 2 bytes for a null terminator) and properly zeroed (or a null byte explicitly appended). Verify all WinAPI calls use the correct (W vs. A) unicode version.

  • I tried adding LA_IN_MEMORY to the flags bitmask and got an error message “Either trial has not started or has been tampered! Trial days left: 30”. This is apparently produced by the next line (IsLicenseGenuine()), meaning that the setProductId() call was successful.
  • Identifying what your native code does differently when the LA_IN_MEMORY flag is not set will probably be very helpful. It’s possible the invalid memory access is associated with identifying the directory or file to be used.
    • There is a recent changelog entry for 3.14.9 involving this flag. Looking at that commit might give a hint to the problem.
    • There’s another recent change in 3.15.0 involving auto-detection of a file on Windows which also may be suspicious given that LA_IN_MEMORY makes the problem go away.
  • When given an invalid key, the error message “43: The product id is incorrect.” is returned, so the point in native code where unowned memory is being accessed is after this error check.

• Trace what is happening with the ID string defined on the Java side. Given the constant definition of the string, the actual memory allocation is likely not a problem, but keep track of the native pointer to this string to be sure it’s not inadvertently overwritten.

As you’ve noted in the comments, reducing the native memory allocation solves this issue, indicating you are hitting a limit. It turns out the default 32-bit Java native memory allocation for stack size (-Xss) is 320 KB. From Oracle docs:

On Windows, the default thread stack size is read from the binary (java.exe). As of Java SE 6, this value is 320k in the 32-bit VM and 1024k in the 64-bit VM.

You can reduce your stack size by running with the -Xss option. For example:

java -server -Xss64k

Note that on some versions of Windows, the OS may round up thread stack sizes using very coarse granularity. If the requested size is less than the default size by 1K or more, the stack size is rounded up to the default; otherwise, the stack size is rounded up to a multiple of 1 MB.

You could increase this limit to solve the problem or, as you’ve indicated in the remarks, lower your native allocation. You might wish to be more conservative than 300K as that only leaves a small amount for other use of the stack. You might also start smaller, check the return value for ERR_MORE_DATA and try again with a larger value. 300KB seems a rather huge amount to devote to registry values.

Note also that 32-bit Java has a total process memory size limit of either 2GB or 4GB, depending on the OS. If your Java heap allocation grows close to that limit, it reduces the native amount available to you. You can control how big the heap gets with the -Xmx switch and you can also ensure sufficient native memory allocation with the -Xss switch. Use these switches in a combination to avoid hitting the process size limit.