lh | 9ed821d | 2023-04-07 01:36:19 -0700 | [diff] [blame^] | 1 | |
| 2 | Implementation notes: |
| 3 | |
| 4 | This is a true OS/400 implementation, not a PASE implementation (for PASE, |
| 5 | use AIX implementation). |
| 6 | |
| 7 | The biggest problem with OS/400 is EBCDIC. Libcurl implements an internal |
| 8 | conversion mechanism, but it has been designed for computers that have a |
| 9 | single native character set. OS/400 default native character set varies |
| 10 | depending on the country for which it has been localized. And more, a job |
| 11 | may dynamically alter its "native" character set. |
| 12 | Several characters that do not have fixed code in EBCDIC variants are |
| 13 | used in libcurl strings. As a consequence, using the existing conversion |
| 14 | mechanism would have lead in a localized binary library - not portable across |
| 15 | countries. |
| 16 | For this reason, and because libcurl was originally designed for ASCII based |
| 17 | operating systems, the current OS/400 implementation uses ASCII as internal |
| 18 | character set. This has been accomplished using the QADRT library and |
| 19 | include files, a C and system procedures ASCII wrapper library. See IBM QADRT |
| 20 | description for more information. |
| 21 | This then results in libcurl being an ASCII library: any function string |
| 22 | argument is taken/returned in ASCII and a C/C++ calling program built around |
| 23 | QADRT may use libcurl functions as on any other platform. |
| 24 | QADRT does not define ASCII wrappers for all C/system procedures: the |
| 25 | OS/400 configuration header file and an additional module (os400sys.c) define |
| 26 | some more of them, that are used by libcurl and that QADRT left out. |
| 27 | To support all the different variants of EBCDIC, non-standard wrapper |
| 28 | procedures have been added to libcurl on OS/400: they provide an additional |
| 29 | CCSID (numeric Coded Character Set ID specific to OS/400) parameter for each |
| 30 | string argument. String values passed to callback procedures are NOT converted, |
| 31 | so text gathered this way is (probably !) ASCII. |
| 32 | |
| 33 | Another OS/400 problem comes from the fact that the last fixed argument of a |
| 34 | vararg procedure may not be of type char, unsigned char, short or unsigned |
| 35 | short. Enums that are internally implemented by the C compiler as one of these |
| 36 | types are also forbidden. Libcurl uses enums as vararg procedure tagfields... |
| 37 | Happily, there is a pragma forcing enums to type "int". The original libcurl |
| 38 | header files are thus altered during build process to use this pragma, in |
| 39 | order to force libcurl enums of being type int (the pragma disposition in use |
| 40 | before inclusion is restored before resuming the including unit compilation). |
| 41 | |
| 42 | Secure socket layer is provided by the IBM GSKit API: unlike other SSL |
| 43 | implementations, GSKit is based on "certificate stores" or keyrings |
| 44 | rather than individual certificate/key files. Certificate stores, as well as |
| 45 | "certificate labels" are managed by external IBM-defined applications. |
| 46 | There are two ways to specify an SSL context: |
| 47 | - By an application identifier. |
| 48 | - By a keyring file pathname and (optionally) certificate label. |
| 49 | To identify an SSL context by application identifier, use option |
| 50 | SETOPT_SSLCERT to specify the application identifier. |
| 51 | To address an SSL context by keyring and certificate label, use CURLOPT_CAINFO |
| 52 | to set-up the keyring pathname, CURLOPT_SSLCERT to define the certificate label |
| 53 | (omitting it will cause the default certificate in keyring to be used) and |
| 54 | CURLOPT_KEYPASSWD to give the keyring password. If SSL is used without |
| 55 | defining any of these options, the default (i.e.: system) keyring is used for |
| 56 | server certificate validation. |
| 57 | |
| 58 | Non-standard EBCDIC wrapper prototypes are defined in an additional header |
| 59 | file: ccsidcurl.h. These should be self-explanatory to an OS/400-aware |
| 60 | designer. CCSID 0 can be used to select the current job's CCSID. |
| 61 | Wrapper procedures with variable arguments are described below: |
| 62 | |
| 63 | _ curl_easy_setopt_ccsid() |
| 64 | Variable arguments are a string pointer and a CCSID (unsigned int) for |
| 65 | options: |
| 66 | CURLOPT_ABSTRACT_UNIX_SOCKET |
| 67 | CURLOPT_CAINFO |
| 68 | CURLOPT_CAPATH |
| 69 | CURLOPT_COOKIE |
| 70 | CURLOPT_COOKIEFILE |
| 71 | CURLOPT_COOKIEJAR |
| 72 | CURLOPT_COOKIELIST |
| 73 | CURLOPT_COPYPOSTFIELDS |
| 74 | CURLOPT_CRLFILE |
| 75 | CURLOPT_CUSTOMREQUEST |
| 76 | CURLOPT_DEFAULT_PROTOCOL |
| 77 | CURLOPT_DNS_SERVERS |
| 78 | CURLOPT_EGDSOCKET |
| 79 | CURLOPT_ENCODING |
| 80 | CURLOPT_FTPPORT |
| 81 | CURLOPT_FTP_ACCOUNT |
| 82 | CURLOPT_FTP_ALTERNATIVE_TO_USER |
| 83 | CURLOPT_INTERFACE |
| 84 | CURLOPT_ISSUERCERT |
| 85 | CURLOPT_KEYPASSWD |
| 86 | CURLOPT_KRBLEVEL |
| 87 | CURLOPT_LOGIN_OPTIONS |
| 88 | CURLOPT_MAIL_AUTH |
| 89 | CURLOPT_MAIL_FROM |
| 90 | CURLOPT_NETRC_FILE |
| 91 | CURLOPT_NOPROXY |
| 92 | CURLOPT_PASSWORD |
| 93 | CURLOPT_PINNEDPUBLICKEY |
| 94 | CURLOPT_PRE_PROXY |
| 95 | CURLOPT_PROXY |
| 96 | CURLOPT_PROXYPASSWORD |
| 97 | CURLOPT_PROXYUSERNAME |
| 98 | CURLOPT_PROXYUSERPWD |
| 99 | CURLOPT_PROXY_CAINFO |
| 100 | CURLOPT_PROXY_CAPATH |
| 101 | CURLOPT_PROXY_CRLFILE |
| 102 | CURLOPT_PROXY_KEYPASSWD |
| 103 | CURLOPT_PROXY_PINNEDPUBLICKEY |
| 104 | CURLOPT_PROXY_SERVICE_NAME |
| 105 | CURLOPT_PROXY_SSLCERT |
| 106 | CURLOPT_PROXY_SSLCERTTYPE |
| 107 | CURLOPT_PROXY_SSLKEY |
| 108 | CURLOPT_PROXY_SSLKEYTYPE |
| 109 | CURLOPT_PROXY_SSL_CIPHER_LIST |
| 110 | CURLOPT_PROXY_TLSAUTH_PASSWORD |
| 111 | CURLOPT_PROXY_TLSAUTH_TYPE |
| 112 | CURLOPT_PROXY_TLSAUTH_USERNAME |
| 113 | CURLOPT_RANDOM_FILE |
| 114 | CURLOPT_RANGE |
| 115 | CURLOPT_REFERER |
| 116 | CURLOPT_RTSP_SESSION_UID |
| 117 | CURLOPT_RTSP_STREAM_URI |
| 118 | CURLOPT_RTSP_TRANSPORT |
| 119 | CURLOPT_SERVICE_NAME |
| 120 | CURLOPT_SOCKS5_GSSAPI_SERVICE |
| 121 | CURLOPT_SSH_HOST_PUBLIC_KEY_MD5 |
| 122 | CURLOPT_SSH_KNOWNHOSTS |
| 123 | CURLOPT_SSH_PRIVATE_KEYFILE |
| 124 | CURLOPT_SSH_PUBLIC_KEYFILE |
| 125 | CURLOPT_SSLCERT |
| 126 | CURLOPT_SSLCERTTYPE |
| 127 | CURLOPT_SSLENGINE |
| 128 | CURLOPT_SSLKEY |
| 129 | CURLOPT_SSLKEYTYPE |
| 130 | CURLOPT_SSL_CIPHER_LIST |
| 131 | CURLOPT_TLSAUTH_PASSWORD |
| 132 | CURLOPT_TLSAUTH_TYPE |
| 133 | CURLOPT_TLSAUTH_USERNAME |
| 134 | CURLOPT_UNIX_SOCKET_PATH |
| 135 | CURLOPT_URL |
| 136 | CURLOPT_USERAGENT |
| 137 | CURLOPT_USERNAME |
| 138 | CURLOPT_USERPWD |
| 139 | CURLOPT_XOAUTH2_BEARER |
| 140 | Else it is the same as for curl_easy_setopt(). |
| 141 | Note that CURLOPT_ERRORBUFFER is not in the list above, since it gives the |
| 142 | address of an (empty) character buffer, not the address of a string. |
| 143 | CURLOPT_POSTFIELDS stores the address of static binary data (of type void *) and |
| 144 | thus is not converted. If CURLOPT_COPYPOSTFIELDS is issued after |
| 145 | CURLOPT_POSTFIELDSIZE != -1, the data size is adjusted according to the |
| 146 | CCSID conversion result length. |
| 147 | |
| 148 | _ curl_formadd_ccsid() |
| 149 | In the variable argument list, string pointers should be followed by a (long) |
| 150 | CCSID for the following options: |
| 151 | CURLFORM_FILENAME |
| 152 | CURLFORM_CONTENTTYPE |
| 153 | CURLFORM_BUFFER |
| 154 | CURLFORM_FILE |
| 155 | CURLFORM_FILECONTENT |
| 156 | CURLFORM_COPYCONTENTS |
| 157 | CURLFORM_COPYNAME |
| 158 | CURLFORM_PTRNAME |
| 159 | If taken from an argument array, an additional array entry must follow each |
| 160 | entry containing one of the above option. This additional entry holds the CCSID |
| 161 | in its value field, and the option field is meaningless. |
| 162 | It is not possible to have a string pointer and its CCSID across a function |
| 163 | parameter/array boundary. |
| 164 | Please note that CURLFORM_PTRCONTENTS and CURLFORM_BUFFERPTR are considered |
| 165 | unconvertible strings and thus are NOT followed by a CCSID. |
| 166 | |
| 167 | _ curl_easy_getinfo_ccsid() |
| 168 | The following options are followed by a 'char * *' and a CCSID. Unlike |
| 169 | curl_easy_getinfo(), the value returned in the pointer should be freed after |
| 170 | use: |
| 171 | CURLINFO_EFFECTIVE_URL |
| 172 | CURLINFO_CONTENT_TYPE |
| 173 | CURLINFO_FTP_ENTRY_PATH |
| 174 | CURLINFO_REDIRECT_URL |
| 175 | CURLINFO_PRIMARY_IP |
| 176 | CURLINFO_RTSP_SESSION_ID |
| 177 | CURLINFO_LOCAL_IP |
| 178 | CURLINFO_SCHEME |
| 179 | Likewise, the following options are followed by a struct curl_slist * * and a |
| 180 | CCSID. |
| 181 | CURLINFO_SSL_ENGINES |
| 182 | CURLINFO_COOKIELIST |
| 183 | Lists returned should be released with curl_slist_free_all() after use. |
| 184 | Option CURLINFO_CERTINFO is followed by a struct curl_certinfo * * and a |
| 185 | CCSID. Returned structures sould be free'ed using curl_certinfo_free_all() after |
| 186 | use. |
| 187 | Other options are processed like in curl_easy_getinfo(). |
| 188 | |
| 189 | _ curl_pushheader_bynum_cssid() and curl_pushheader_byname_ccsid() |
| 190 | Although the prototypes are self-explanatory, the returned string pointer |
| 191 | should be freed after use, as opposite to the non-ccsid versions of these |
| 192 | procedures. |
| 193 | Please note that HTTP2 is not (yet) implemented on OS/400, thus these |
| 194 | functions will always return NULL. |
| 195 | |
| 196 | |
| 197 | Standard compilation environment does support neither autotools nor make; |
| 198 | in fact, very few common utilities are available. As a consequence, the |
| 199 | config-os400.h has been coded manually and the compilation scripts are |
| 200 | a set of shell scripts stored in subdirectory packages/OS400. |
| 201 | |
| 202 | The "curl" command and the test environment are currently not supported on |
| 203 | OS/400. |
| 204 | |
| 205 | |
| 206 | Protocols currently implemented on OS/400: |
| 207 | _ DICT |
| 208 | _ FILE |
| 209 | _ FTP |
| 210 | _ FTPS |
| 211 | _ FTP with secure transmission |
| 212 | _ GOPHER |
| 213 | _ HTTP |
| 214 | _ HTTPS |
| 215 | _ IMAP |
| 216 | _ IMAPS |
| 217 | _ IMAP with secure transmission |
| 218 | _ LDAP |
| 219 | _ POP3 |
| 220 | _ POP3S |
| 221 | _ POP3 with secure transmission |
| 222 | _ RTSP |
| 223 | _ SCP if libssh2 is enabled |
| 224 | _ SFTP if libssh2 is enabled |
| 225 | _ SMTP |
| 226 | _ SMTPS |
| 227 | _ SMTP with secure transmission |
| 228 | _ TELNET |
| 229 | _ TFTP |
| 230 | |
| 231 | |
| 232 | |
| 233 | Compiling on OS/400: |
| 234 | |
| 235 | These instructions targets people who knows about OS/400, compiling, IFS and |
| 236 | archive extraction. Do not ask questions about these subjects if you're not |
| 237 | familiar with. |
| 238 | |
| 239 | _ As a prerequisite, QADRT development environment must be installed. |
| 240 | _ If data compression has to be supported, ZLIB development environment must |
| 241 | be installed. |
| 242 | _ Likewise, if SCP and SFTP protocols have to be compiled in, LIBSSH2 |
| 243 | developent environment must be installed. |
| 244 | _ Install the curl source directory in IFS. |
| 245 | _ Enter shell (QSH) |
| 246 | _ Change current directory to the curl installation directory |
| 247 | _ Change current directory to ./packages/OS400 |
| 248 | _ Edit file iniscript.sh. You may want to change tunable configuration |
| 249 | parameters, like debug info generation, optimisation level, listing option, |
| 250 | target library, ZLIB/LIBSSH2 availability and location, etc. |
| 251 | _ Copy any file in the current directory to makelog (i.e.: |
| 252 | cp initscript.sh makelog): this is intended to create the makelog file with |
| 253 | an ASCII CCSID! |
| 254 | _ Enter the command "sh makefile.sh > makelog 2>&1' |
| 255 | _ Examine the makelog file to check for compilation errors. |
| 256 | |
| 257 | Leaving file initscript.sh unchanged, this will produce the following OS/400 |
| 258 | objects: |
| 259 | _ Library CURL. All other objects will be stored in this library. |
| 260 | _ Modules for all libcurl units. |
| 261 | _ Binding directory CURL_A, to be used at calling program link time for |
| 262 | statically binding the modules (specify BNDSRVPGM(QADRTTS QGLDCLNT QGLDBRDR) |
| 263 | when creating a program using CURL_A). |
| 264 | _ Service program CURL.<soname>, where <soname> is extracted from the |
| 265 | lib/Makefile.am VERSION variable. To be used at calling program run-time |
| 266 | when this program has dynamically bound curl at link time. |
| 267 | _ Binding directory CURL. To be used to dynamically bind libcurl when linking a |
| 268 | calling program. |
| 269 | _ Source file H. It contains all the include members needed to compile a C/C++ |
| 270 | module using libcurl, and an ILE/RPG /copy member for support in this |
| 271 | language. |
| 272 | _ Standard C/C++ libcurl include members in file H. |
| 273 | _ CCSIDCURL member in file H. This defines the non-standard EBCDIC wrappers for |
| 274 | C and C++. |
| 275 | _ CURL.INC member in file H. This defines everything needed by an ILE/RPG |
| 276 | program using libcurl. |
| 277 | _ LIBxxx modules and programs. Although the test environment is not supported |
| 278 | on OS/400, the libcurl test programs are compiled for manual tests. |
| 279 | _ IFS directory /curl/include/curl containing the C header files for IFS source |
| 280 | C/C++ compilation and curl.inc.rpgle for IFS source ILE/RPG compilation. |
| 281 | |
| 282 | |
| 283 | |
| 284 | Special programming consideration: |
| 285 | |
| 286 | QADRT being used, the following points must be considered: |
| 287 | _ If static binding is used, service program QADRTTS must be linked too. |
| 288 | _ The EBCDIC CCSID used by QADRT is 37 by default, NOT THE JOB'S CCSID. If |
| 289 | another EBCDIC CCSID is required, it must be set via a locale through a call |
| 290 | to setlocale_a (QADRT's setlocale() ASCII wrapper) with category LC_ALL or |
| 291 | LC_CTYPE, or by setting environment variable QADRT_ENV_LOCALE to the locale |
| 292 | object path before executing the program. |
| 293 | _ Do not use original source include files unless you know what you are doing. |
| 294 | Use the installed members instead (in /QSYS.LIB/CURL.LIB/H.FILE and |
| 295 | /curl/include/curl). |
| 296 | |
| 297 | |
| 298 | |
| 299 | ILE/RPG support: |
| 300 | |
| 301 | Since 95% of the OS/400 programmers use ILE/RPG exclusively, a definition |
| 302 | /INCLUDE member is provided for this language. To include all libcurl |
| 303 | definitions in an ILE/RPG module, line |
| 304 | |
| 305 | h bnddir('CURL/CURL') |
| 306 | |
| 307 | must figure in the program header, and line |
| 308 | |
| 309 | d/include curl/h,curl.inc |
| 310 | |
| 311 | in the global data section of the module's source code. |
| 312 | |
| 313 | No vararg procedure support exists in ILE/RPG: for this reason, the following |
| 314 | considerations apply: |
| 315 | _ Procedures curl_easy_setopt_long(), curl_easy_setopt_object(), |
| 316 | curl_easy_setopt_function() and curl_easy_setopt_offset() are all alias |
| 317 | prototypes to curl_easy_setopt(), but with different parameter lists. |
| 318 | _ Procedures curl_easy_getinfo_string(), curl_easy_getinfo_long(), |
| 319 | curl_easy_getinfo_double() and curl_easy_getinfo_slist() are all alias |
| 320 | prototypes to curl_easy_getinfo(), but with different parameter lists. |
| 321 | _ Procedures curl_multi_setopt_long(), curl_multi_setopt_object(), |
| 322 | curl_multi_setopt_function() and curl_multi_setopt_offset() are all alias |
| 323 | prototypes to curl_multi_setopt(), but with different parameter lists. |
| 324 | _ The prototype of procedure curl_formadd() allows specifying a pointer option |
| 325 | and the CURLFORM_END option. This makes possible to use an option array |
| 326 | without any additional definition. If some specific incompatible argument |
| 327 | list is used in the ILE/RPG program, the latter must define a specialised |
| 328 | alias. The same applies to curl_formadd_ccsid() too. |
| 329 | |
| 330 | Since RPG cannot cast a long to a pointer, procedure curl_form_long_value() |
| 331 | is provided for that purpose: this allows storing a long value in the curl_forms |
| 332 | array. |