How can create own section in memory

//mysection.c: this program tells how we can create own section and also put our varible inside this section
#include

int my_func (int, int) __attribute__ ((section ("my_code_section")));
int global_var __attribute__ ((section ("my_data_section")));
int global_init __attribute__ ((section ("my_data_section"))) = 2;

int my_func (int i, int j)
{
return i*j;
}

int
main (void)
{
int local_var = 10;
global_init = 5;

printf ("local_var: %d global_var: %d global_init: %d\n",
local_var, global_var, global_init);
printf ("%d * %d = %d\n", local_var, global_var,
my_func (local_var, global_var));

return 0;
}
$gcc mysection.c
$./a.out
local_var: 10 global_var: 0 global_init: 5
10 * 0 = 0

How to call a function before main and after main

1. Using GCC compiler on Linux/Solaris

/*main.c :this program tells how we can call a function before main() call and after main() call*/

#include

void my_ctor (void) __attribute__ ((constructor));
void my_dtor (void) __attribute__ ((destructor));

void
my_ctor (void)
{
printf ("hello before main()\n");
}

void
my_dtor (void)
{
printf ("bye after main()\n");
}

int
main (void)
{
printf ("hello\n");
return 0;
}

$gcc main.c
$./a.out
hello before main()
hello
bye after main()


2. Using CC compiler on Solaris[Using Pragma]

Using pragma
============

#pragma init (foo)

void foo()
{
printf("\nInside foo");
}

#pragma fini (bar)

void bar()
{
printf("\nInside bar");
}

int
main()
{
printf("\nInside main");
return 0;
}

$cc main.c
$./a.out
Inside foo
Inside main
Inside bar

How we can call a function after exiting from main() Using ATEXIT()

//atexit.c: this program shows how we can call a function after exiting from main() function
#include
#include

void my_func1(void);
void my_func2(void);

int main(int argc, char** argv)
{
atexit(my_func1);
atexit(my_func2);
printf("Inside main\n");
exit(0);

}

void
my_func1(void)
{
printf("\ninside my_func1 ");
}

void
my_func2(void)
{
printf("\ninside my_func2 ");
}


$gcc atexit.c
$./a.out

Inside main
inside my_func2
inside my_func1

Linker Puzzles

When static linker resolve the symbols at link time as follows rules applied:
Note: Linker always search for the symbol name from the symbol table.
Rule1: Multiple strong symbols are not allowed
Rule2: Given a strong symbol and multiple weak symbols, choose the strong symbol
Rule3: Given multiple weak symbols, choose any of the weak symbols

What is strong symbol and Weak symbol?
--- Strong symbols are those that have initialized.
exampls:
int x = 2;
float y = 3.2;
function()
{ }

---Weak symbols are those that are uninitialized.
examples:
int x;
float y;
char a;
static int a;

Now solve the as follows linker puzzles based on above information

1.
/*Module 1*/
int main()
{
return 0;
}

/*Module 2*/
static int main=1;
int f1()

2.
/*Module 1*/
int x;
int main()
{ return 0;
}


/*Module 2*/
float x;
int f1()
{
}

3.
/*Module 1*/
int x=1;
void main()
{
}

/*Module 2*/
int x;
4.
/*Module 1*/
int x=1;
void main()
{
}

/*Module 2*/
float x=1.0;


5.
/*Module 1*/
int x=1;
void main()
{
}

/*Module 2*/
int x =2;

Answers:
1. No error successfully build since main in module1 is strong but in module2 this is weak(as declared static)
2. No Error pic any one.
3. No error take strong symbol from module1.
4. Error like
ld: fatal: symbol `x' is multiply-defined:
(file /var/tmp//ccnY5GzB.o type=OBJT; file /var/tmp//ccfHCTBb.o type=OBJT);
ld: fatal: File processing errors. No output written to a.out
collect2: ld returned 1 exit status


5. Error like
ld: fatal: symbol `x' is multiply-defined:
(file /var/tmp//ccnY5GzB.o type=OBJT; file /var/tmp//ccfHCTBb.o type=OBJT);
ld: fatal: File processing errors. No output written to a.out
collect2: ld returned 1 exit status

How to run a shared library on Linux

In my prevoius blog I have written how to run the shared libraries on Open-Solaris.
http://bhushanverma.blogspot.com/2008/06/how-to-run-shared-library-on-open.html

Shared object should have following entries to run:
1. +x permission that is by default is given by the static linker(program linker) when creating a shared object.
2. Entry point at which the program/shared library is starts to run.
3. Interpreter(Run time linker) that is used to run any shared library after loaded by kernel part exec().

Entry point at which the program/shared library is starts to run can be
given by passing -Wl,-e entry_point to the linker at command line:

To create .interp section by using GNU gcc, use the follwing line of code on linux:
const char my_interp[] __attribute__((section(".interp"))) = "/lib/ld-linux.so.2";

Where /lib/ld-linux.so.2 is the path of interpreter(Run time linker)　 in linux.

In open solaris we passed -Wl,-I,/usr/lib/ld.so.1 to the sun linker to create this section.
I think in gnu linker this option is available but do other things.

Demo on Linux machine:
-------------------------
$ cat func.c
const char my_interp[] __attribute__((section(".interp"))) = "/lib/ld-linux.so.2";
#include
void bar();

int
func()
{
printf("Hacking\n");
bar();
exit (0);
}

void
bar()
{
printf("Bye...\n");
}

$ gcc -fPIC -o func.so -shared -Wl,-e,func func.c

You can see that foo.so have .interp section and interp program header.
# readelf -l func.so
Elf file type is DYN (Shared object file)
Entry point 0x4dc
There are 7 program headers, starting at offset 52

Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
PHDR 0x000034 0x00000034 0x00000034 0x000e0 0x000e0 R E 0x4
INTERP 0x0005a3 0x000005a3 0x000005a3 0x00013 0x00013 R 0x1
[Requesting program interpreter: /lib/ld-linux.so.2]
LOAD 0x000000 0x00000000 0x00000000 0x005bc 0x005bc R E 0x1000
LOAD 0x0005bc 0x000015bc 0x000015bc 0x00104 0x0010c RW 0x1000
DYNAMIC 0x0005d4 0x000015d4 0x000015d4 0x000c0 0x000c0 RW 0x4
NOTE 0x000114 0x00000114 0x00000114 0x00024 0x00024 R 0x4
GNU_STACK 0x000000 0x00000000 0x00000000 0x00000 0x00000 RW 0x4

Section to Segment mapping:
Segment Sections...
00
01 .interp
02 .note.gnu.build-id .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rel.dyn .rel.plt .init .plt .text .fini .rodata .interp .eh_frame
03 .ctors .dtors .jcr .data.rel.ro .dynamic .got .got.plt .bss
04 .dynamic
05 .note.gnu.build-id
06

You can cleary see, func.so have .interp section and INTERP program header.
Now try to run func.so:
$ ./func.so
Hacking
Bye...


hua.. thats cool.
Happy hacking.

Accessing libc,runtime linker,shared libraries functions using pragma weak

Sometimes you are working on libc, glibc linker ,shared libraries.
You have seen in the code of these libraries as follows:
#pragma weak dlopen = _dlopen
#pragma weak dlcose= _dlclose

What is this? what is the use of these lines of code written in libc,glibc etc.
The above lines code indicates that any user applications can call/use "dlopen" and "dlclose"
functions.

How is this possible?
There is little trick/work done by compiler, which creates the
symbol as a weak.

#pragma weak symbol1 = symbol2
This pragma declares symbol1 to be a weak alias of symbol2. It is an error if symbol2 is not defined in the
current translation unit.
for more information please refers following link.
http://gcc.gnu.org/onlinedocs/gcc/Pragmas.html#Pragmas

Now we have some idea on weak symbol,so move forwards to do some R&D.
I have done this R&D on Open-Soalris. Its also applicable to Linux.

$cat mylibc.c

#pragma weak my_function = _my_function
void _my_function()
{

printf("inside _my_fnction()\n");

}

$ cat main.c

int
main()
{
my_function();
return 0;

}

Now build the shared library:
$ gcc -o mylibc.so -fpic -shared mylibc.c
$ elfdump -s mylibc.so |fgrep my_function
[1] 0x0000049c 0x0000002f FUNC WEAK D 0 .text my_function
[10] 0x0000049c 0x0000002f FUNC GLOB D 0 .text _my_function
[49] 0x0000049c 0x0000002f FUNC WEAK D 0 .text my_function
[58] 0x0000049c 0x0000002f FUNC GLOB D 0 .text _my_function

You can see here my_function as a weak symbol.

Now build the executable using mylibc.so as a dependency:
$ gcc -o main main.c ./mylibc.so
$ elfdump -s main |fgrep my_function
[27] 0x08050680 0x00000000 FUNC GLOB D 0 UNDEF my_function
[81] 0x08050680 0x00000000 FUNC GLOB D 0 UNDEF my_function

Now run the executable:
$ ./main
inside _my_function

Cool! we got the result, Now try yourself.
Happy Hacking.

TLS(Thread Local Storage)

Thread-local storage (TLS) as the name indicate is a method to make static or global memory local to a thread.
Separate copies of thread-local data that have been allocated at compile-time, must be associated with individual threads of execution.
This is sometimes needed because all threads in a process share the same address space. In other words,data in a static or global variable is normally always located at the same memory location, when referred to by threads from the same process.
Local Variables are already local to threads, because each thread has its own stack, residing in a different memory location.So TLS is not applicable to local variables.

Take an example how two threads worked and update a common global variable:
If thread A sets the thread local variable to 1, and thread B then sets it to 2, then code running in thread A will continue to see the value 1 for the variable while code running in thread B sees the value 2. In Posix threads this type of variable can be created via pthread_key_create() and accessed via pthread_getspecific() and pthread_setspecific().

These functions work well , but making a function call for each access is awkward and inconvenient. It would be more useful if you could just declare a regular global variable and mark it as thread local. That is the idea of Thread Local Storage (TLS).On a system which supports TLS, any global (or static) variable may be annotated with __thread. The variable is then thread local.

To create and use TLS requires following supports:
1. The compiler.
2. Program linker(Static linker)
3. dynamic linker(Run Time Linker) and some kernel support is also needed(thread library).

The design of TLS on ELF systems fully supports shared libraries, including having multiple shared libraries, and the executable itself, use the same name to refer to a single TLS variable. TLS variables can be initialized. Programs can take the address of a TLS variable, and pass the pointers between threads, so the address of a TLS variable is a dynamic value and must be globally unique.


Following are some Compilers list that are able to create TLS:
Sun Studio C/C++, IBM XL C/C++, GNU C & Intel C/C++
all the aboove compilers uses __thread keyword to create thread local storgae.
Variables are declared thread-local using the __thread keyword, as in the following examples:
__thread int i;
__thread static int j;
__thread char *ptr;




Thread-Local Storage Access Models
---------------------------------------------
Each TLS reference follows one of the following access models.
Defining different storage models for TLS variables.

* Local Executable(Staic TLS): Permits access to TLS variables defined in the executable itself.
This model can only reference TLS variables which are part of the TLS block of the dynamic
executable. The link-editor calculates the thread pointer-relative offsets statically, without the
need for dynamic relocations, or the extra reference to the GOT. This model can not be used to
reference variables outside of the dynamic executable.

* Initial Executable: Permits access to a variable which is known to be part of the TLS image of the executable.
This is true for all TLS variables defined in the executable itself, and for all TLS variables in shared libraries explicitly linked with the executable. This is not true for accesses from a shared library, nor for accesses to TLS variables defined in shared libraries opened by dlopen.

* General(Global) Dynamic: Fully general access to TLS variables from an executable or a shared object.

* Local Dynamic: Permits access to a variable which is bound locally within the executable or shared object from which it is referenced. This is true for all static TLS variables, for example. It is also true for protected symbols.

Following examples shows how to make executable and shared object using these models
1. Local Exec model

Example:
--------

//executable itself
__thread int i;

int
foo()
{
return(i);
}

int
main()
{
foo();
}

2.Initial Exec model

Example:can be in executable or in shared object
-------
extern __thread int i;

int
foo()
{
retun i;
}


3.General (Global)Dynamic

Example:
--------

__thread int i;

int
foo()
{
return(i);
}

4.Local Dynamic

Example:
--------

static __thread int i,j;

int
foo()
{
return (i + j);
}

-----------


For more information Please read:
http://docs.sun.com/app/docs/doc/817-1984/chapter8-1?a=view

Linker and Loader

Runtime Linker,Static Linker and Loader
===============================

Concept of Run Time Linker and Static Linker is same in case of Solaris and Linux.

There are two linkers:

・ A static linker (also called link-editor) that you use to link your application binaries after
compiling your source code into object files

・ A dynamic linker (also called as Run time linker or Interpreter) performing the run time linking of dynamic executable and shared libraries.
The runtime linker plays an important role in the execution of the program.
The dynamic linker is the runtime linker (also called loader or ld.so.1(1)). Since this loaded shared libraries also load executable file if not loaded by exec().

In Open-Solaris Dynamic linker is ld.so.1 that resides in /usr/lib/ld.so.1, which is soft link (symbolic link) to /lib/ld.so.1
In Linux Dynamic linker name is ld-linux.so.2 that resides in /lib/ld-linux.so.2

The first kind of linker is the link-editor (also called linker or static linker or native linker or program linker ie ld, and you use it to link object files into shared objects or executable. The user can either directly execute the link-editor, or it can be invoked by the compiler.

Please do not invoke link-editor directly ,if you use linker directly .init and .fini section will not registered properly.

By default, the static linker makes all applications' symbols global in scope (making them into what is also called "exported symbols"). This means it puts the symbols into the dynamic symbol table of the resulting binary such that other binary modules can access those symbols.

dynamic relocations that the dynamic linker performs are only necessary for the global (also known as external or exported) symbols. The static linker resolves references to local symbols (for example, names of static functions) statically when it links the binary.


Linking Stages
============

There are 3 types of linking in case of Solaris,Linux,Windows.
1. Static Linking
-------------------
Static linking means that for each function your program calls, the assembly to that function is actually included in the executable file. Function calls are performed by calling the address of this code directly, the same way that functions of your program are called.

2. Dynamic Linking
-----------------------
Dynamic Linking means there are shared libraries which are used by application.
This linking is done at compile and build time of an executable.
The required relocations of data and functions are done at load and run time by Run Time Linker.
3. Runtime Linking
-----------------------
Runtime linking is linking that happens when a program calls a function from a library that is was not linked against at compile time. The library is mapped with dlopen() under Solaris ,Linux,UNIX, and LoadLibrary() under Microsoft Windows, both of which return a handle that is then passed to symbol resolution functions (dlsym() and GetProcAddress()), which actually return a function pointer that may be called directly from the program as if it were any normal function.


There can be two type of executable
============================
1. Static executable :This type of executable is build using static linking as described above.

What happen when running the static executable:
-----------------------------------------------
The kernel ie exec() would load the executable and jump directly to its entry point(ehdr:e_entry) ie _start address used by linker. Static means "no interpreter (ie no run time linker)".
2.Dynamic executable:
This type of executable is build using dynamic linking or run time linking as described above.

What happen when running the dynamic executable:
------------------------------------------------
exec() (ie part of the kernel) works as loader and do the following:
----------------------------------------------------------------
1. exec() loads the executable.
2. exec() inspects the executable to find the interpreter name.
3. exec() loads the interpreter(run time linker).
4. exec() jumps to the interpreter.

The interpreter(run time linker)
--------------------------------
1.Analyse and loads the executable dependencies.
2.relocates any loaded objects
3.and then jumps to the executable.
The Interpreter
-------------------
An executable file may have one PT_INTERP program header element if executable file not contains PT_INTERP section Then this is Static Executable.Dynamic Executable always contains PT_INTERP section in which the path of run time linker ] is written.
During exec ie loading executable, the system retrieves a path name from the PT_INTERP segment and creates the initial process image from the interpreter file's segments. That is, instead of using the original executable file's segment images, the system composes a memory image for the interpreter.
It then is the interpreter's responsibility to receive control from the system and provide an environment for the application program.
The interpreter receives control in one of two way:
-----------------------------------------------------------
1. First, it may receive a file descriptor to read the executable file, positioned at
the beginning. It can use this file descriptor to read and/or map the
executable file's segments into memory.
2. Second, depending on the executable file format, the system may load the executable file into
memory instead of giving the interpreter an open file descriptor.
An interpreter may be either a shared object or an executable file. In Linux or Solaris this is a shared object.

ELF

I worked on ELF, so writing some basic details about this.
Here I will use　gcc,　elfdump on Open-Solaris for explanation.
You can done whatever I am doing here on linux using GNU toolchain and objdump.

What is ELF?
ELF means Executable and Linking Format this is executable format which is used now a days in UNIX variant platforms such as Linux,Solaris etc.
In other way wen say ,It defines the format of executable binaries used on Linux,Solaris etc - and also for relocatable, shared object and core dump files too.

ELF is used by both Static Linkers,Dynamic Linker and loaders. For more information on Static Linkers,Dynamic Linker and Loaders, please read my following bolg:

http://bhushanverma.blogspot.com/search/label/ELF%2FLinker%2FLoader

Executable and Linkable Format is:

Standard library format for object files
Derived from AT&T System V Unix
Later adopted by BSD Unix variants and Linux
Generic name: ELF binaries
Better support for shared libraries than old a.out formats.

ELF describes three types of Object File:

• relocatable file(.o)
• executable file
• shared object file(.so)

--------------

Now moves from Theory to Practical
How We can see inside ELF:

Step1: Make an c source file as
$cat test.c

int
main()
{
return 0;
}

Step 2:Build executable as
$gcc -o main main.c

Step 3: Now fire following command
$elfdump -e main
foll lowing will be displayed
ELF Header
ei_magic: { 0x7f, E, L, F }
ei_class: ELFCLASS32 ei_data: ELFDATA2LSB
e_machine: EM_ARM e_version: EV_CURRENT
e_type: ET_EXEC
e_flags: 514
e_entry: 0x8360 e_ehsize: 52 e_shstrndx: 14
e_shoff: 0x544 e_shentsize: 40 e_shnum: 17
e_phoff: 0x34 e_phentsize: 32 e_phnum: 5


Lets try to understand one by one of this header elements

ei_magic:This is magic number of ELF ie this tell about file is ELF or not.
ei_class:This tells about class types ie 32 bit file or 64 bit file.
ELFCLASS32:32 bit file
ELFCLASS64:64 bit file
ei_data: This tell about endianness.ie this run on Little-endian machine or Big-endian machine.
ELFDATA2LSB means Litte -endian
ELFDATA2MSB means Big-endian
e_machine: This tells about for which this file is build or on which machine this executable will run.
e_version: This member identifies the object file version.
e_type: This tells about type of file is executable,object,shared object or core
Executable -- e_type:ET_EXEC
Shared Object(.so) -- e_type:ET_DYN
Relocatable object files (.o)-- e_type:ET_REL

e_entry: Its entry point ie _start address used by linker.This gives the virtual address to which the system first transfers control,for starting the process.
e_shoff: This holds the section header table's file offset in bytes. If the file has no section header table, this holds zero.
e_ehsize: This holds the ELF header's size in bytes.
e_shstrndx: This holds the section header table index of the entry associated with the
section name string table. If the file has no section name string table, this member
holds the value SHN_UNDEF.
e_shnum:This holds the number of entries in the section header table.
e_phoff: This holds the program header table's file offset in bytes.
e_phentsize: This holds the size in bytes of one entry in the file's program header table;
all entries are the same size.
e_phnum:This holds the number of entries in the program header table

The above is only one example to read the ELF file. There are plenty information inside ELF file.
There are various tools to play with ELF.
If you want to play with ELF use the following tools:
GNU Binutils:
* ar - A utility for creating, modifying and extracting from archives.
* nm - Lists symbols from object files.
* objcopy - Copys and translates object files.
* objdump - Displays information from object files.
* ranlib - Generates an index to the contents of an archive.
* readelf - Displays information from any ELF format object file.
* size - Lists the section sizes of an object or archive file.
* strings - Lists printable strings from files.
* strip - Discards symbols.

Open-Solaris Tools:
elfdump:Displays information from any ELF format object file.
nm: Lists symbols from object files.


Views of ELF:
There are two views: Linking View and Execution View
While doing low level programming know that ELF files consist of several sections.
for example,
elfdump -c main shows
----------------------------------------------------------
Section Header[1]: sh_name: .interp
sh_addr: 0x80d4 sh_flags: [ SHF_ALLOC ]
sh_size: 0x11 sh_type: [ SHT_PROGBITS ]
sh_offset: 0xd4 sh_entsize: 0
sh_link: 0 sh_info: 0
sh_addralign: 0x1

Section Header[2]: sh_name: .hash
sh_addr: 0x80e8 sh_flags: [ SHF_ALLOC ]
sh_size: 0x54 sh_type: [ SHT_HASH ]
sh_offset: 0xe8 sh_entsize: 0x4
sh_link: 3 sh_info: 0
sh_addralign: 0x4


....
Section Header[16]: sh_name: .strtab
sh_addr: 0 sh_flags: 0
sh_size: 0xe0 sh_type: [ SHT_STRTAB ]
sh_offset: 0xb0c sh_entsize: 0
sh_link: 0 sh_info: 0
sh_addralign: 0x1
----------------------------------------------------------------

ELF loader in the kernel doesn't really have any relation with ELF sections at all, instead it looks on so called "program headers".

elfdump -p main shows program headers as
--------------------------------------------------------------
Program Header[0]:
p_vaddr: 0x8034 p_flags: [ PF_X PF_R ]
p_paddr: 0x8034 p_type: [ PT_PHDR ]
p_filesz: 0xa0 p_memsz: 0xa0
p_offset: 0x34 p_align: 0x4

Program Header[1]:
p_vaddr: 0x80d4 p_flags: [ PF_R ]
p_paddr: 0x80d4 p_type: [ PT_INTERP ]
p_filesz: 0x11 p_memsz: 0x11
p_offset: 0xd4 p_align: 0x1

Program Header[2]:
p_vaddr: 0x8000 p_flags: [ PF_X PF_R ]
p_paddr: 0x8000 p_type: [ PT_LOAD ]
p_filesz: 0x3d0 p_memsz: 0x3d0
p_offset: 0 p_align: 0x8000

Program Header[3]:
p_vaddr: 0x103e0 p_flags: [ PF_W PF_R ]
p_paddr: 0x103e0 p_type: [ PT_LOAD ]
p_filesz: 0xc0 p_memsz: 0xc0
p_offset: 0x3e0 p_align: 0x8000

Program Header[4]:
p_vaddr: 0x103e0 p_flags: [ PF_W PF_R ]
p_paddr: 0x103e0 p_type: [ PT_DYNAMIC ]
p_filesz: 0x98 p_memsz: 0x98
p_offset: 0x3e0 p_align: 0x4


ELF files has two views:
1. Section view (Linking View) defining what goes where in file
2. Program view (Execution View) defining exact mapping of ELF data in process address space.
A Program header table tells the system how to create a process image.
Files used to build a process image (execute a program) must have a program header table;
Relocatable files do not need one.
A section header table contains information describing the file's sections.






Figure ELF views





Have a Nice Day! Keep Enjoy .

Bhushan