IBMMQ consumer application unable to consume TextMessage ( JMSCMQ1049: The character set ‘1208(UTF-8) Unmappable Action: REPORT)

I have a consumer application that uses IBMMQ to consume messages from the queue manager. I have no control over the publisher, only the consumer. Here is the part of the code for my consumer:

while(true) {
    message = (TextMessage) queueReceiver.receive(200);
                
    if (message != null) {
             messageFile = messageTypeToListenerMapping.get(type).generateMessageNameAndWriteToDisk(message.getText(), source, saveDirectory);
    } else {
             break;
    }      

This code works fine and is able to retrieve the message as a TextMessage object for almost all of my queues except for one queue. When the application attempts to retrieve a message from this queue, I get this error:

2021-10-05 22:12:31.831 DEBUG 30050 --- [nio-8181-exec-8] o.s.web.servlet.DispatcherServlet        : GET "/retrieveAllMessagesFromQueue", parameters={}
2021-10-05 22:12:31.832 DEBUG 30050 --- [nio-8181-exec-8] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped to public java.util.Map<java.lang.String, java.util.Map<java.lang.String, java.util.Map<java.lang.String, java.lang.Integer>>>  ,
2021-10-05 22:12:36.882 DEBUG 30050 --- [nio-8181-exec-8] .m.m.a.ExceptionHandlerExceptionResolver : Using @ExceptionHandler public Exception.ApiErrorResponse Octopus.controller.OctopusRestController.handleJmsException(javax.jms.JMSException)
2021-10-05 22:12:36.899 DEBUG 30050 --- [nio-8181-exec-8] m.m.a.RequestResponseBodyMethodProcessor : Using 'application/json', given [*/*] and supported [application/json, application/*+json, application/json, application/*+json]
2021-10-05 22:12:36.899 DEBUG 30050 --- [nio-8181-exec-8] m.m.a.RequestResponseBodyMethodProcessor : Writing [com.ApiErrorResponse@4be14d5e]
2021-10-05 22:12:36.900 DEBUG 30050 --- [nio-8181-exec-8] .m.m.a.ExceptionHandlerExceptionResolver : Resolved [com.ibm.msg.client.jms.DetailedJMSException: JMSCMQ1049: The character set '1208(UTF-8) Unmappable Action: REPORT, Unmappable Replacement: 63, spaceByte: 32' cannot convert some or all of the string '[B@47412132'
An attempt was made to send or receive string data using a character set not capable of translating the strings content.
Only encode a message using a character set known to be appropriate for the string data being transmitted.]
2021-10-05 22:12:36.901 DEBUG 30050 --- [nio-8181-exec-8] o.s.web.servlet.DispatcherServlet        : Completed 500 INTERNAL_SERVER_ERROR

After doing some debugging, I found that the error occurs when this code is executed: message.getText()

I believe that perhaps the message being put into the queue by the publisher contains some sort of special characters which the application is unable to handle. So I tried to see if I could consume the message for this specific queue using BytesMessage instead. Here is the code below:

BytesMessage byteMessage;

byteMessage = (BytesMessage) queueReceiver.receive(200);
int TEXT_LENGTH = new Long(byteMessage.getBodyLength()).intValue();
byte[] text_bytes = new byte[TEXT_LENGTH];
byteMessage.readBytes(text_bytes, TEXT_LENGTH);
String codePage = byteMessage.getStringProperty(WMQConstants.JMS_IBM_CHARACTER_SET);
String textString = new String(text_bytes, codePage);

     if (textString != null) {
              messageFile = messageTypeToListenerMapping.get(type).generateMessageNameAndWriteToDisk(textString, source, saveDirectory)
     } else {
              break;
     }

However I am coming across this error:

2021-10-27 22:29:34.046 DEBUG 20987 --- [nio-8181-exec-1] o.s.web.servlet.DispatcherServlet        : Failed to complete request: java.lang.ClassCastException: com.ibm.jms.JMSTextMessage cannot be cast to javax.jms.BytesMessage
2021-10-27 22:29:34.053 ERROR 20987 --- [nio-8181-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.ClassCastException: com.ibm.jms.JMSTextMessage cannot be cast to javax.jms.BytesMessage] with root cause
java.lang.ClassCastException: com.ibm.jms.JMSTextMessage cannot be cast to javax.jms.BytesMessage

From what I understand, the message cannot be casted from TextMessage to ByteMessage, which also means that the publisher sent the message as a TextMessage. If the publisher sent the message as a TextMessage, then why am I coming across this issue in the first place? I am a bit confused with all of this and would appreciate some guidance.

---

EDIT: I have ran amqsbcg as requested by @MoragHughson and here are the details to one of the messages:

 MQGET of message number 14861, CompCode:0 Reason:0
****Message descriptor****

  StrucId  : 'MD  '  Version : 2
  Report   : 0  MsgType : 8
  Expiry   : -1  Feedback : 0
  Encoding : 546  CodedCharSetId : 1208
  Format : 'MQSTR   '
  Priority : 0  Persistence : 1
  MsgId : X'414D5120514D2E44303014D5120514D2E443030A0EF121'
  CorrelId : X'000000000000000000000000000000000000000000000000'
  BackoutCount : 0
  ReplyToQ       : '                                                '
  ReplyToQMgr    : 'QM.1010101_4642                                 '
  ** Identity Context
  UserIdentifier : 'user2      '
  AccountingToken :
   X'0A31303030303639333331639333331639333331004568490000000000000006'
  ApplIdentityData : '                                '
  ** Origin Context
  PutApplType    : '6'
  PutApplName    : 'mqm_sender             '
  PutDate  : '00000000'    PutTime  : '00000000'
  ApplOriginData : '    '

  GroupId : X'000000000000000000000000000000000000000000000000'
  MsgSeqNumber   : '1'
  Offset         : '0'
  MsgFlags       : '0'
  OriginalLength : '-1'

****   Message      ****

 length - 64 of 64 bytes

00000000:  0000 0000 B3DC B059 6400 0000 524D 5320           '.....ܰYd...RMS '
00000010:  2052 3235 3139 3435 3734 3437 3030 3030           ' R25194574470000'
00000020:  3120 2020 2020 2020 2020 2020 2020 2032           '1              2'
00000030:  3032 3131 3032 3231 3431 3333 3730 3020           '021102214133700 '



 No more messages
 MQCLOSE

It appears that the CodedCharSetId is 1208, which seems to be UTF-8. If that is the case, then TextMessages should work fine if i am not mistaken?

Answer

So, you have an interesting problem & I’ve been down this road before.

It is the opposite of what George Costanza from Friends would say: “it’s me not you”. In your case, it is the sender and not you.

There are 3 important pieces of information from the amqsbcg dump:

Encoding : 546  CodedCharSetId : 1208
Format : 'MQSTR   '
00000000:  0000 0000 B3DC B059 6400 0000 524D 5320           '.....?Yd...RMS '

(1) That message is not a JMS (aka MQRFH2) message but JMS/MQ layer will turn it into a JMS message for the application.

• If the MQMD Format field is set to “MQSTR ” then the MQ/JMS layer will create a TextMessage.
• If the MQMD Format field is set to ” ” (all blank) then the MQ/JMS layer will create a BytesMessage.

(2) Since the incoming message’s MQMD Format field is set to “MQSTR “, the JMS/MQ layer will attempt to convert it from the CCSID of 1208 to the CCSID of your JVM. The problem comes from the fact that you have non-alphanumeric data in your message. Lots of binary zeroes in the first 12 bytes.

You have 2 options:

(1) Ask the sender to stop including non-alphanumeric data in the message.

(2) If the non-alphanumeric data is important then tell the sender to set the message’s MQMD Format field to ” ” (all blank). Then when you get the message, you simply cast it as BytesMessage.

Most likely some junior programmer copied code from StackOverflow or somewhere else and used it without understanding what the code did.

Note to future MQ developers/programmers: If your message payload has non-alphanumeric data then never ever set the message’s MQMD Format field to “MQSTR “.