=============================================================================== Notices =============================================================================== * The runtime option (environment variable) NMPI_IB_RNDV_PROTOCOL=GET or RGET cannot be used. If it is explicitly set, disable it. * Starting NEC MPI-2.27.0 and NEC MPI-3.6.0, MPI compile commands issue a warning for the linking specified with the -static option. Using the compiler option -static is not recommended with MPI compile commands. MPI programs require shared system libraries and shared library for MPI memory management to execute, so MPI compile commands append -Wl,-Bdynamic to the end of the command line to force dynamic linking. The mix of -Wl,-Bdynamic appended by MPI compile commands and -static may lead unexpected behavior. If you want to link MPI program against static libraries, please surrounded the libraries with -Wl,-Bstatic and -Wl,-Bdynamic instead of the -static option. * Part of MPI dynamic process creation and management functions are not available. The functions are as follows - MPI_CLOSE_PORT - MPI_COMM_ACCEPT - MPI_COMM_CONNECT - MPI_COMM_JOIN - MPI_OPEN_PORT - MPI_LOOKUP_NAME - MPI_PUBLISH_NAME - MPI_UNPUBLISH_NAME * If you use MPI-IO to access the same file, all processes must be access the file through the filesystems having the same type. Otherwise, incorrect result may occur. * MPI programs cannot run between models with EDR HCAs and models with HDR/NDR HCA. * Currently, SHARP is known to be unavailable on the following combinations of MLNX_OFED and ConnectX Firmware. In this case, the communication using SHARP is unavailable even for NEC MPI. - MLNX_OFED 23.04-1.1.3.0 and ConnectX-7 Firmware 28.38.1002 - MLNX_OFED 23.04-1.1.3.0 and ConnectX-7 Firmware 28.38.0458 - MLNX_OFED 5.8-1.1.2.1 and ConnectX-7 Firmware 28.38.1002 - MLNX_OFED 5.8-1.1.2.1 and ConnectX-7 Firmware 28.38.0458 * There are two series of NEC MPI, Version 2 and Version 3. The two series support different MLNX_OFED and VE, but except for the features related to these, the two series provide equivalent functionality. Version 2: Supports MLNX_OFED 4.x and VE10/VE10E/VE20 Version 3: Supports MLNX_OFED 5.x or later and VE10/VE10E/VE20/VE30 The first release of Version 3, Version 3.0.0, is equivalent to Version 2.21.0, Thereafter, Version 3.x.y and Version 2..y will provide equivalent functionality. Each version series of NEC MPI does not guarantee the results when running in MLNX_OFED environments other than the above versions, even if InfiniBand is not used. * When using the traceback function with VE1 binaries running on VE30, use NEC MPI Version 3.4.0 or later, and NEC Compiler Version 5.0.1 or later. Otherwise, the traceback may not be output properly. * With NEC MPI Version 3.4.0 and 2.25.0, if mpincc, mpinc++ or mpinfort compile command is used, it is no longer necessary to escape special characters ";", "&", "|", and "`" in command line options. * If you are using NEC MPI Version 3.3.0, you must not specify the -lm option explicitly when linking a VE3 program that performs long double type division with the -static option. The program so generated will terminate abnormally with "Unable to grow stack" at runtime. Since the NEC compiler implicitly adds libm, the -lm option need not be specified explicitly. * With NEC MPI Version 3.3.0 and 2.24.0, if mpincc, mpinc++ or mpinfort compile command is used, it is no longer necessary to escape special characters "(", ")", "<", and ">" in command line options. * For supporting performance counters of VE30, when running MPI programs on VE30 specifying VE_PERF_MODE=VECTOR-MEM, the performance items of NMPI_PROGINF=DETAIL are changed as follows: The MPI libraries of NEC MPI Version 3.3.0 or later VLD LLC Hit Element Ratio (%) and Actual Load B/F are output instead of Actual V. Load B/F The MPI libraries of NEC MPI Version 3.2.0 or before VLD LLC Hit Element Ratio (%) is output instead of Actual V. Load B/F * Enhanced PPS support of NEC MPI Version 3.2.0 and 2.23.0 is available on the following environment: - VEOS Version 2.14.1 or later - NQSV R1.12-186 or later - necmpi job of NQSV and the queue that hydra is selected as NEC MPI Process Manager (also interactive queue is included) * NEC MPI 2.x.x and NEC MPI 3.x.x have the following notes because some communication functions are incompatible between MLNX_OFED 4.x and MLNX_OFED 5.x or later. - The MPI executable that statically links the NEC MPI 2.x.x library for MLNX_OFED 4.x can only be run in MLNX_OFED 4.x environment. - The MPI executable that statically links the NEC MPI 3.x.x library for MLNX_OFED 5.x or later can only be run in MLNX_OFED 5.x or later environment. Therefore, in MLNX_OFED 5.x or later environment, a MPI executable file statically linked with the NEC MPI 2.x.x library cannot be run. The NEC MPI 3.x.x library must be relinked in order to run in MLNX_OFED 5.x or later environment. Note that MPI executable files dynamically linked with NEC MPI 2.x.x libraries can run in the MLNX_OFED 5.x or later environment by changing to NEC MPI 3.y.y or later libraries with the same functions as 2.x.x at runtime. * Enhanced PPS support of NEC MPI Version 2.20.0 is effective, only if it is used together with VEOS Version 2.10 or later and NQSV Version 1.09 or later. * If non-blocking MPI-IO procedures are used in NEC MPI Version 2.20.0 or later, it is recommended to use VEOS 2.10.1 or later for the performance improvement. * Functions related to NQSV HYDRA process manager require the following software. - NQSV R1.08-471 or later - NEC MPI Version 2.18.0 or later In a NQSV request submitted to a batch queue that HYDRA is selected as NEC MPI Process Manager, if NEC MPI Version 2.17.0 or before is used, the output of the MPI program is set as the standard output and standard error output of the MPI execution command. * Performance improvement of VH-VE hybrid MPI for IB-less system becomes effective with VEOS Version 2.8 or later. * NMPI_VE_TRACEBACK=ON requires the following compilers. - SDK C/C++ compiler Version 3.1.1 or later - SDK Fortran compiler Version 3.1.1 or later (when using Fortran) * If you compile object files and applications with NEC MPI-2.11.0, please recompile them with NEC MPI-2.12.0 or later. * Non Swappable Memory Size Used of MPI runtime performance information (NMPI_PROGINF) requires VEOS Version 2.7.2 or later. * VEOS v2.4.1 or later is recommended for MPI communication optimization on Model A412-8 or B401-8. * mpi_f08 requires SDK Fortran compiler Version 2.4.1 or later. * NEC MPI requires the following software. - SDK C/C++ compiler Version 2.2.2 or later - SDK Fortran compiler Version 2.2.2 or later (when using Fortran) - SDK Tuning Tool PROGINF/FTRACE Version 2.1.0 or later (SDK: NEC Software Development Kit for Vector Engine)