RFC destination fails with error "Incomplete Logon Data"

On newly built systems all RFC destinations fail with the following error:

The transport Management system may also report the error
XT101 "RFC error when sending logon data".


Reproducing the issue may result in the following being written to the corresponding work process trace file:
dev_wX:
N RSEC: The entry with
identifier
/HMAC_INDEP/RFC_EXTERNAL_TICKET_4_TRUSTED_SYSTEM
N was encrypted by a system
N with different SID and cannot be decrypted here.
M *** ERROR => LocFunc_ReadKeyFromDB(): Error in function:
'RSecDReadItem()' (errorcode=-2)


The solution is contained in * Note: 1532825 Deleting SECSTORE entries during system export/system copy.

Entries with error message “Error when opening an RFC connection” in transaction SM58

Purpose

The Purpose of this page is to provide further information regarding the error messages “error when opening an RFC connection” with tRFC entries (transaction /SM58).

Overview

This page explains the reason for the tRFC failures and how to get rid of them.

The error cause

The "Error when opening an RFC connection" error message is a result of the affected RFC destination (in this example, the TSTCLNT100) not working when an entry was being processed. This might happen due to intermittent network glitches or any network issue.

The solution

First of all you need to ensure that the RFC destination affected (TSTCLNT100) is working fine (results of the connection test):

Then, if there is no error with the RFC destination, you’ll be able to reprocess the failed entry manually (in Transaction /SM58 => Edit => Execute LUW). You can also schedule the report RSARFCEX to run at regular intervals in your landscape.  The report RSARFCEX will reprocess any failed entry in the tRFC layer. Therefore it is always recommended that this report run at very regular intervals (for example every 2 minutes) to reprocess RFC requests which might have failed due to an intermittent network glitch. 

Test connection in SM59 fails with the error "Invalid Length. Check Parameters"

Known problems:

1615847 SM59 connection test fails


The following error will be reported:





The dev_rfc trace will report something like the following:



The error will occur if there is a problem resolving the sapgwXX port.

• Check the etc/services file for incorrect port configuration
• Check for duplicate entries in etc/services file
• Check if a firewall between client and server is causing the issue

Remote Logon button in SM59 does not work

Known problems:

189077 The remote login in SM59 does not work
51367 RFC logon under different user or client



Clicking on the Remote Logon button in SM59 returns no response:




It is necessary to ensure that the source and destination hosts should be able to successfully perform the two-way hostname<->IP lookup for the other host that is involved in the RFC connection.


Problems have occurred whereby the response from the RFC-call receiver to the sender gets stuck.

The sending system determines his own IP-address via the command'hostname' on operating system level. This IP-address is send within the RFC protocol to the receiving system. NAT and other IP-address translations in the firewall are not working on RFC level. The receiver sends a response back to this IP-address, so this IP-Address has to be known on receiver side.
Solution:

• Don't use IP-address translations in your firewall. Use firewall and saprouter instead, if possible
• Define a IP-address routing on the receiving system to route the IP-address to the correct address.

QRFCTRACE table contains an excessive amount of entries

There is a function module that is used to delete the entries from table QRFCTRACE called 'QRFC_TRACE_DELETE'.
It is possible for customers to write an ABAP program that can be executed in the background and will call this function module.
The other possibility is to delete the trace information in each tRFC queue name separately. When the user is prompted to delete the trace information, you can enter the name of
the queue instead of entering '*'.
Transaction SMQE:






There have been problems whereby the FM QRFC_TRACE_DELETE ends with a time-out error. In this case, it is possible to executed a "truncate" command on QRFCTRACE table.
If you want to stop the table form being populated,you can deactivate the trace via SMQ1>QRFC> Trace> Deactivate Trace.
If you deactivate the trace, it should stop collecting data in table QRFCTRACE.

Related note:

706478 Preventing Basis tables from increasing considerably

CPIC Return Code 679 - Transaction Program not Registered

Communication error 679 is reported in the system log (SM21):





All CPIC return codes are mentioned in note 63347. Return code 679 means "Transaction Program is not registered":

TP_NOTREGISTERED 679 TP not registered
To find out more detail on which program is causing the error the corresponding rfc trace file must be checked.

In the example above, the work process number (Nr) is 01 so the dev_rfc1 (St11) will contain more details. Search this file for the timestamp reported in sm21, in this case it is 171012.
You will see something like the following:





This shows that the problem is related to RFC destination REGTEST and program "Program1". If you
go to transaction SM59 and test the destination REGTEST the "program not registered" error will
be returned:





Note 353597 provides details on how to register a server program:

<program name> -a<program ID> -g<SAP gateway hostname> -x<gatewayservice>

So, for our example above "Program1 "needs to be registered:

<program name> -aProgram1 -gy52tdc00 -xsapgw52

<program name> is the name of the program you are trying to run.
Program1 is the name given in SM59 (ProgramId).
Y52tdc52 is the gateway host (from the dev_rfc1 trace above) and sapgw52 is the gateway service (from the dev_rfc1 trace).

If the program has been registered on the local gateway you should see an entry for it in
SMGW ==> "Logged on Clients", it will be shown as a REGISTERED_TP / REGISTERED SERVER.

The example given in note 353597 is:

rfcexec -atest -gHOST -xsapgw00

Please note that "rfcexec" is a sample program delivered with the RFCSDK. If you do the following:

rfcexec -aProgram1 -gy52tdc00 -xsapgw52

This means that you are trying to register the sample program "rfcexec" with ID "Program1" on the gateway.

If you are not trying to connect to program rfcexec, the developer of the program you are trying to run should be contacted to advise how to correctly register their program.
In some cases, a particular service may need to be started.

Many background ARFCXXXX jobs visible in SM37

Background jobs with the name "ARFC:<XXXX>" can be seen in SM37:





The job logs shows that these jobs are running report RSARFCSE:





The job RSARFCSE is responsible for asynchronous remote function calls
that have not been succesful.


For every unsuccesful ARFC found in table ARFCSSTATE a job called ARFC:<xxxxxxx> is created.
In table ARFCSSTATE you can see the destination for the ARFC call.


Once you have determined the target system for the ARFC calls (as per
table ARFCSSTATE), use transaction SM59 to check whether this is a valid RFC destination.


It is possible to disable these background jobs if errors occur with the
connection. To do this, goto the "tRFC options "for the relevant RFC
destination reporting the connection error:





In the next screen choose the "Supress background job if conn.error" option:




The RFC's can then be reprocessed by scheduling report
RSARFCEX. You can also delete the already scheduled jobs with report RSBTCDEL
if there are too many.
Using the option "Supress background job if conn. error" will stop the
creation of these ARFC:<XXXX> background jobs. You can then reprocess
the failed tRFC calls by scheduling report RSARFCEX at regular intervals.

Deleting unwanted entries from RFC tables (ARFCSSTATE, ARFCSDATA etc.)

Known problems:

375566 Large number of entries in tRFC and qRFC tables
366869 HOLD/EXECUTED/WCONFIRM entries in ARFCRSTATE
1621114  RSTRFCEH - Performance improvement with deletion
For tRFC outbound the tables used are ARFCSSTATE and ARFCSSDATA.
For qRFC outbound the tables used are TRFCQOUT ARFCSSTATE and ARFCSSDATA.
The RFC tables should be kept as small as possible - in some cases these tables can contain millions of entries.
To check the number of entries goto SE16, enter the table name, for example ARFCSDATA, then choose the "Number of Entries" Option:

The 3 notes above detail reports for deleting entries in the RFC tables.
Note that report RSTRFCEF is not to be used anymore, we have a new report to check inconsistencies as described in the SAP note: 779664
The report RSTRFCEF will be replaced by the report RSTRFCEG.
Note that report RSTRFCEG contains a parameter CHECK_ONLY, if you mark it, then the inconsistencies will not be deleted.
The standard way to delete queue entries is to run report RSTRFCQD/DS as a batchjob as mentioned in note
763255
In SMQ1 you can goto "QRFC" in the Menu and then choose reorganise, this will delete ALL queues in SMQ1. However, if you want to delete
selected queues then you could choose "Edit" in the menu and then choose "delete Selected objects".

Inbound Scheduler (SMQR) is in status WAIT

Known problems:

742950 Performance affected on Oracle DB with Supplement 11
744869 Optimizing the performance of inbound scheduler
697884 Queues in SMQ2 are not processed
683459 qRFC performance improvements in the inbound scheduler

1493644 Long running process reading TRFCQIN blocking execution
1497510 Experiencing deadlocks, lock & waits in inbound processing
1500048 SMQ2 inbound Queue remains in READY state
This problem can occur of there are no resources available for qRFC processing.
To check this, use trnx "SMQR" goto "QRFC Resources", you may see something
like the following:

This shows that there are 7 dialogs available for qRFC processing.
However, problems may occur of there are no dialog avaiable for some or
all servers. In this case you would see something like
the folllowing:

This can occur if no dialogs have been assigned for the AS group used in SMQR. To resolved the problem
allocate resources via trnx RZ12.
The Scheduler may remain in status WAIT if the system cannot cope with the load. To check this, goto transaction SM51, choose "goto", "Server Information" and "Queue Information":

This will then show a screen similar to the following:

If the "requests waiting" value is close to the "Max Req waiting value" this shows a resource issue as the system cannot cope with the load. In this case, the customer may need to add extra dialog work processes or balance the load over more application servers.
Dialogs may also be unavailable if the connection table for one or all servers are full. In this case, you will see the error "GwFiCreateConvId table overflow" in SM21 - to resolve the issue see note 1180734 Overflow of communication table.
Status WAIT can also occur if there are many application dependancies - in this case smq2 would show many queues in status waiting - waiting on other queues. SMQ2 should show something like the following:


If there are many dependancies the relevant application should advise further - maybe the number of dependancies could be reduced. In the example above the relevant application would be CRM.

Outbound Scheduler (SMQS) remains in status WAITING

Known problems:

977283    Outbound scheduler remains in WAITING status for a long time
1051445  qRFC scheduler does not use all available resourcen
1485789  QRFC: Long running processes in SM50
1484197  ARFCSDATA for one LUW was read by 100 DIA processes
1528988  Wrong Index in Queue Scheduler when accessing TRFCQOUT
1483757  Slow processing tRFC, qRFC in a high load environment


SMQS will show the following:




A. Other than the two notes mentioned above the issue may be caused because of a performance issue reading the RFC tables. This happens only using Oracle databases.
In this case you would see problems doing a Sequential read in SM50 on RFC tables such as TRFCQOUT, ARFCSSTATE or ARFCSSDATA, this can beseen in SM50 or SM66.
If so, note 742950 needs to be applied: Performance affected on Oracle DB with Supplement 11
Solution
o You are using an Oracle database Version 9l or higher.
Proceed as described in Note 932975
o You are using an Oracle database that is lower than 9l.
After you import Supplement 11, you must run the RSTRFCCS report.
Change the default settings to 5000 LUWs (NLUW), so that the
program creates different STATUS values in TRFCQOUT and TRFCQIN.
Only with these settings, the INDEX on the STATUS field of the
TRFCQIN and TRFCQOUT tables is correctly used. These entries are
deleted again after the statistics are generated.
B.
The problem can also be caused if there ar not enough resources available.
In this case check trnx SMQS, choose "goto" in the menu and then choose "QRFC Resources". Resources should be available for tRFC and qRFC processing:





If NO resources are available for tRFC and qRFC processing you will see something like:





If this is the case, dialogs will need to be assigned to the Application Server group mentioned in SMQS.
The settings can be changed in RZ12.
However, if dialogs have already been assigned in RZ12 but are not visible in SMQS "Goto" "Resources" then the problem may be that the communication table for the particular server is full.
If so, see note 1180734 Overflow of communication table



The Scheduler may remain in status WAIT if the system cannot cope with the load. To check this, goto transaction SM51, choose "goto", "Server Information" and "Queue Information":





This will then show a screen similar to the following:





If the "requests waiting" value is close to the "Max Req waiting value" this shows a resource issue as the system cannot cope with the load. In this case, you will need to add extra dialog work processes or balance the load over more application servers.

RFC Connections remain in SM04

Known problems:

Note 900132 Remaining user contexts, remaining locks
Note 1261669 RFC connections are not closed
Note 864455 SosGetAnchor error message relating to SNC and BEx
Note 1660720 Session remains open after the logoff on enterprise portal.
If you suspect RFC connections are not being logged off check SM04, it should show many RFC connections



The last request time will be shown here but to verify the last request date choose the relevant entry in SM04 then goto "User" in the Menu and then "Technical Information":





The date and time of the last request will be shown in the field modeinfo[0].diatime, for example:





Note 568271 describes the "lifetime of an RFC connection".
The note indicates that whoever opens the RFC connection must also close it. It is up to the application that opened the connection to close it when it has finished.
According to this note it is the responsibility of RFC caller to close the connection. A client program opens the connection for calling funtion module(s) and has to close this connection,
as soon as this connection is in use any more.
On the R/3 side the connections are kept for the total time where the client transaction is active.
Most applications choose not to do any special handling and so the system defaults to keeping the RFC connection
open as long as the context (ie. transaction) is active to avoid re-authentication and session establishment.
Thus, RFC connections are held until a transaction is finished. This is the way that RFC connections were designed to work. The RFC connectiondoes not close until the transaction is exited.
This is because of performance issues, since setting up a connection and its server side session is time/space consuming.
For external RFC clients, such as C- or visual basic programs, the programmer has to close the connection via RfcClose function. If the client side opens then connection it is also upto them to close
the connection.
To find out exactly which function module is being used Note 780817 "Analyzing the RFC user in SM04" is very useful
If problems occur when using the Visual Composer please check the solution on note 1254158
Visual Composer doesn't close RFC connections with backend.


In relation to portal connectons please see the following part of note 1507034:
"RFC connections from the Portal will remain after logoff The sessions in the backend remain alive while the connection remains in the free connections j2ee pool.
To reduce the time that the connections remain open, you can decrease the connection lifetime following SAP Note 809954:
Go to Visual Admin -> server -> services -> Connector Container ->
com.sapportals.connectors.sap -> SAPFactory -> Managed Connection
Factory -> Connection Definition, and set the Connection Lifetime to,
for example 60 (or even less).

This will mean that after 60 seconds, open connections will be released
from the pool and the session in the backend will be closed (do not set
Connection Lifetime to 0, it means the Connection lifetime never
expires)."

Entries in Transaction SM58 are in status "Transaction recorded"

Purpose

The purpose of this wiki is to provide tips on troubleshooting  "Transaction recorded"  entries in SM58.

Overview

This page demonstrates with screenshots how to identify and  troubleshoot " Transaction recorded" issues in SM58.

Identifying the issue

A screen similar to the following will be shown in SM58:


Check SMQS to see if destination NBP800 is registered on the outbound scheduler for tRFC processing:

To do this goto SMQS find the relevant destination and click on the value in the TYPE field - usually it is R for Registered:




Here we can see that destination NBP800 is registered for both qRFC and tRFC processing.

Troubleshooting

If entries are remaining in SM58 in status "transaction recorded" and the destination is regsitered on the outbound scheduler for tRFC processing, the only way to speed up the processing of these entries is by increasing the "max conn" value for that particular destination in SMQS. If destination is not registered in SMQS for trfc processing the entries in SM58 can be reprocessed by scheduling report RSARFCEX.
Report RSARFCEX will not work for the entries where the destination is registered for tRFC processing.

The number of max connections can be seen in SMQS ALSO:
Trnx SMQS:




Destination NBP800 is Registered (Type "R") on the Outbound scheduler. The "Max. conn." Value is 1 which means that the maximum number of used dialog used for this destination is 1, this may cause a problem so the number can be increased.
To do this, highlight the destination and choose "Edit" and "Registration":





You will see the following:




"Max conn" value can be changed here.
If you are increasing the max conn value, check that there are enough resources available.
To do this from SMQS, choose "goto" in the Menu and then "qRFC Resources":


You will see something like:


Make sure that there are enough resources for tRFC/qRFC processing. If the value is set to 0 or if the "qRFC Resources" show "not ok"
the parameters in note
74141 will need to be set to allocate resources for trfc/qRFC processing.

"Transaction recorded" usually happens when A. processing idocs or B. BW loads.
A.If it occurs when processing idocs you will see function module "IDOC_INBOUND_ASYNCH" mentioned in SM58.
Check also that the idocs are being processed in the background and not in the foreground.
Ensure that background processing is used for ALE communications.
Report RSEOUT00 (outbound)can be configured to run very specifically for the high volume message types on their system. Schedule regular runs of report RESOUT00 it can be run for
IDoc Type and\or Partner etc..
To set to background processing for Outbound idocs do the following:
-> go to transaction WE20 -> Select Partner Select Outbound Message Type and change the processing method from
"Transfer IDoc Immedi." to "Collect IDocs".
B.If "transaction recorded" occurs when processing BW loads the function module "RSAR_TRFC_DATA_RECEIVED" will be seen in SM58, check also note 916706 Number of dialog processes for data transfer.  In this case, also check transaction SM66 on the BW side, if you see many work processes in error running report SAPLSENA then this needs to be checked by BW application colleagues.