No | Case | Difference Point | Reason | |
1 | Floating point exceptions. | Floating-point exceptions related to sub-normal numbers could not be raised by VE architecture but x86 architecture supports it well. | VE Architecture does not support sub-normal numbers. | |
2 | Minimum stack size for thread, i.e. PTHREAD_STACK_MIN | In VE,
minimum stack size for thread is 4MB. So in VE-glibc the corresponding macro
value is updated, While it is 16KB in glibc for x86 architecture. |
Customized for VE architecture. | |
3 | Specifying user-defined thread stack with pthread_attr_setstack() | If
user creates a thread using user-defined thread stack with
pthread_attr_setstack() API, stack overflow cannot be detected. So it is strongly recommended not to use pthread_attr_setstack(). The user can use pthread_attr_setstacksize() to specify stack size of child thread. The use of pthread_attr_setstacksize() will not cause above stated issue. |
As
per the threading APIs man-pages, following is the behaviour: When the user allocates the thread stack using pthread_attr_setstack(), then the guard size attribute is ignored (i.e. no guard area is created by the library). It is the application's responsibility to handle stack overflow (perhaps by using mprotect(2) to manually define a guard area at the end of the stack that it has allocated). In VEOS, the behaviour is different. Even after providing a guard area (user provided guard), the stack would continue to grow beyond the end of stack and would result in undesirable behaviour. Hence the API pthread_attr_setstack() should not be used. When a user specify stack size of child threads, use pthread_attr_setstacksize(3). pthread_attr_setstacksize() doesn't cause above mentioned issue |
|
4 | Variable $ORIGIN | In
VE(libc), if the user program has
reference to the variable "$ORIGIN" eg. dlopen
("$ORIGIN/XXX.so", RTLD_LAZY) then its value will be fetched from
the environment variable "VE_LD_ORIGIN_PATH" whereas X86 will fill
its value from "readlink" system call Note:- The users are requested to set environment variable "VE_LD_ORIGIN_PATH" before running their program if they intend to specify the directory containing the library (or program) through the use of $ORIGIN variable in their program. |
Readlink system call gives the path of ve_exec instead of VE binary when "/proc/self/exe" is passed as an argument. | |
5 | Loading "ld-linux-ve.so.1" directly from "ve_exec" or "execve()" | We cannot explicitly load VE binary through dynamic linker whereas X86 dynamic linker "ld-linux-x86-64.so.2" could load X86 binary when given as argument. | Current loader implementation in VEOS does not support loading of shared library. | |
6 | "ldd" command | "ldd" for VE displays the dependency list only for binaries and not shared libraries whereas "ldd" for x86_64 displays the dependency list for both. | This internally invokes dynamic linker "ld-linux-ve.so.1" which again is not supported in VE (ref #5) | |
7 | Improvement of memory allocation by malloc() | a)
For VE, DEFAULT_MMAP_THRESHOLD_MAX macro set to 1GB, while it is 32MB for x86 architecture in
Glibc. |
malloc()
allocates memory using sbrk() or mmap(), depending on size. If malloc()
request is below
"mmap_threshold" memory is allocated using sbrk(), else it is
allocated using mmap(). But VE has very large page size (2MB and 64MB), and there is more memory wastage when memory is allocated using mmp() system call for malloc() request size above "mmap_threshold". Because size is always aligned to page size. To avoid this DEFAULT_MMAP_THRESHOLD is set to 1GB, So that for all malloc() requests below 1GB, memory will be allocated using sbrk() only. |
|
b) for VE, DEFAULT_MMAP_THRESHOLD macro set to 1GB, while it is 128 KB for x86 architecture in Glibc | ||||
c) For VE, Maximum number of arena created by glibc malloc() for any process is set to 1, while it multiple of number of available cores for x86 architecture in glibc. | For
VE HEAP_MAX_SIZE is set to 2GB and therefore per thread arena size is also
2GB (Glibc malloc() maintains separate heap for each thread, we called it as
per thread Arena). In multithreaded application whenever thread request small amount of memory using malloc () for first time per thread arena of size 2GB is created, which results in memory wastage if there are multiple threads (each thread will block 2GB of memory physical memory). To avoid the memory wastage number of maximum arena created by glibc malloc() for any process is set to one. So that only on arena will be used by all threads. This difference can be observed by user in pmap output of VE process. Using mallopt() user can change M_MMAP_THRESHOLD between 0 to 1GB for VE architecture, while on x86 Architecture it can be change between 0 to 32MB. |
|||
d) For VE, HEAP_MAX_SIZE macro value changed to 2GB. while it is 64MB for x86 architecture in Glibc. | In
Glibc HEAP_MAX_SIZE is twice of
DEFAULT_MMAP_THRESHOLD_MAX). This difference can be observed by user in pmap output of VE process. |
|||
8 | (VE_) prefix in glibc's environment variables | a)
Environment variables related to glibc mallopt() (MALLOC_xxx) are started
with "VE" prefix. e.g.
MALLOC_ARENA_TEST variable for VE architecture will become
VE_MALLOC_ARENA_TEST, while in x86 it is use without any prefix. The Environment variabales used by VE process with VE-Glibc are. VE_MALLOC_ARENA_MAX VE_MALLOC_ARENA_TEST VE_MALLOC_CHECK_ VE_MALLOC_MMAP_MAX_ VE_MALLOC_MMAP_THRESHOLD_ VE_MALLOC_PERTURB_ VE_MALLOC_TRIM_THRESHOLD_ VE_MALLOC_TOP_PAD_ |
VEOS
runs on VH (x86) and if same name is used for environment variables related
to malloc() and memusage. then variable exported for VH process will also affect the behaviour of VE process or vice versa. To avoid this environment variables name for VE processes should start with "VE" prefix. |
|
b)
Environment variables related to memusage command (MEMUSAGE_xxx) are started with "VE" prefix.
e.g. MEMUSAGE_OUTPUT variable for VE architecture will become VE_MEMUSAGE_OUTPUT, while in x86 it is use without any prefix. The Environment variabales used by VE process with VE-Glibc are. VE_MEMUSAGE_PROG_NAME VE_MEMUSAGE_OUTPUT VE_MEMUSAGE_BUFFER_SIZE VE_MEMUSAGE_NO_TIMER VE_MEMUSAGE_TRACE_MMAP |
||||
c)Other environment variables used by Ve-Glibc
programs are as under: VE_LD_WARN VE_LD_DEBUG VE_LD_AUDIT VE_LD_VERBOSE VE_LD_PRELOAD VE_LD_PROFILE VE_LD_BIND_NOW VE_LD_BIND_NOT VE_LD_SHOW_AUXV VE_LD_HWCAP_MASK VE_LD_ORIGIN_PATH VE_LD_LIBRARY_PATH VE_LD_DEBUG_OUTPUT VE_LD_DYNAMIC_WEAK VE_LD_USE_LOAD_BIAS VE_LD_POINTER_GUARD VE_LD_PROFILE_OUTPUT VE_LD_TRACE_PRELINKING VE_LD_TRACE_LOADED_OBJECTS VE_GCONV_PATH VE_GETTEXT_LOG_UNTRANSLATED VE_LANGUAGE VE_OUTPUT_CHARSET VE_LOCPATH VE_I18NPATH VE_GETCONF_DIR VE_SOTRUSS_FROMLIST VE_SOTRUSS_EXIT VE_SOTRUSS_WHICH VE_SOTRUSS_OUTNAME VE_SOTRUSS_TOLIST VE_MALLOC_TRACE VE_MALLOC_CHECK_ Whereas X86 programs recognize these variables without "VE_" prefix |
||||
9 | Profiling for VE | 1.
Time value is not correct especially when context switch occurs in VE 2. User cannot use SIGALRM in application along with gprof in VE |
In
glibc(x86_64) ,"setitimer()"
uses "ITIMER_PROF" flag for
profiling , but in glibc(VE) , "setitimer()" uses
"ITIMER_REAL" flag for profiling , as in VE architecture
"ITIMER_PROF" flag is not supported. #For more details regarding setitimer() please refer "3. Partially Supported System Calls " in "Difference Points for System Calls" document. |
|
10 | system() API | In
VE, the environment variable set/unset within the program(VE) are not carried
forward to the child program invoked through system() API. Scenario: User exports any environment variable from the command prompt and the same environment variable is unset (using unsetenv) in the main program before calling a child process through system() where the child program intends to use the unset environment variable. In this scenario, the child program see the environment variable exported from the command prompt. |
For VE process, system() request is executed by VH OS (Linux) instead of VEOS. The child program doesn't carry forward the updated environment variable because the environment variable in VE is not reflected in VH environment. | |
11 | system() API | In VE architecture when a VE process invokes system() API for loading and executing a new VE program. All the signal handlers for the newly created VE task will be reset to SIG_DFL. | system()
API implementation for VE creates a new VH process. This newly created VH
process will further load new VE binary by invoking VH execve() with VE
binary as an argument. This will ultimately invoke ve_exec for loading new VE
binary. The ve_exec request to VEOS will create completely new VE process which doesn’t have parent child relation with VE process which has invoked the system () API. Hence all the signal handler routine of the newly created process will be reset to SIG_DFL. |
|
12 | pthread_detach( ) API | Detaching with a thread that has previously been joined by pthread_join() results in an undefined behavior. | In VE, if the thread has already been joined using "pthread_join()" and the same thread is being detached subsequently using "pthread_detach( )" then this results in undefined behavior because, ideally, the pthread structure has already been deallocated at the time of pthread_join(). However, glibc of VH maintains caching and sometimes preserves the thread-id based on certain factors. Hence, glibc of VH may return an error when a thread that has previously been joined by pthread_join() is detached by pthread_detach(). | |
Note:- | ||||
The difference point regarding system calls of VEOS and Linux is already captured in "Difference Points for System Calls" document. |