Messages

1

Unix / Linux Discussion / Re: Who has the most RAM ?

« on: September 28, 2005, 04:22:33 pm »

"doesn't support more than 960MB of RAM"

It's not 960mb, and the ram is used, just not in the kernel address space.

2

Unix / Linux Discussion / Re: Who has the most RAM ?

« on: September 28, 2005, 01:40:59 pm »

Alright! Enough fun. Now, I don't want to be a dick but I don't actually care how much RAM any of you have. The truth is that this is another lesson-post in trap form!

EDIT -
tmp, may I ask why we need more than 900MB of RAM?

Good question, Joe! I had a sneaking suspicion that many of you guys probably have some fancy machines (with plenty o' RAM), but you probably didn't know that Linux, out of the box anyways, doesn't support more than 960MB of RAM ;P Don't believe me ? Check out Newby's situation ;P

newby@impaler:~$ free -m
             total       used       free     shared    buffers     cached
Mem:           883        850         32          0        265        103
-/+ buffers/cache:        481        401
Swap:          933         65        868

It's 1024MB RAM, but...

BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
 BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 000000003ff30000 (usable)
 BIOS-e820: 000000003ff30000 - 000000003ff40000 (ACPI data)
 BIOS-e820: 000000003ff40000 - 000000003fff0000 (ACPI NVS)
 BIOS-e820: 000000003fff0000 - 0000000040000000 (reserved)
 BIOS-e820: 00000000ffb80000 - 0000000100000000 (reserved)
Warning only 896MB will be used.

If you ran free on a shell server (cheated) then you probably got the expected amount because the admin hopefully knows what he's doing.

Anyhow, on with the issue at hand.

Good news: I'm about to tell you how to fix the problem

Bad news: You'll have to recompile your kernel.

The linux kernel actually supports up to 64GB of RAM. However, in order to use anything past 960MB you need to configure  the CONFIG_NOHIGHMEM define appropriately then rebuild your kernel. You want something like so:

up to 960MB: off
up to 4GB: 4GB
up to 64GB: 64GB

If you select 64GB your processor must be capable of PAE mode. That includes all processors since the Pentium Pro, so you're probably OK. The kernel will refuse to boot if you don't have it which is why this configuration isn't enabled by default.

Also, if you've got an older BIOS it might have trouble reporting the amount of RAM to the bootloader accurately. If you've taken the above step and it still doesn't work properly add this to your /etc/lilo.conf:

Code: [Select]

append="mem=2048M"

Where I have 2 Gigs of RAM. All should be well.

I think your post is a little misleading, I'll try to see if I can clear the concept up, from what I remember reading in "Understanding the Linux Kernel"

First of all, the processor can directly access 4GB of virtual memory, because it has a 32 bit address space. (Assuming we are using a 32 bit processor) This window is split into two types: user address space, and the kernel address space.

These two are separated by the PAGE_OFFSET values, which I know to be 0xC0000000 under linux. This means that under the default, 3gb is allocated to the user address space, and 1gb is allocated to the kernel.

The kernel has a special area of memory reserved for vmalloc, and it dedicates 128mb to it. That means the kernel can only use 896 mb of mapped memory directly, since 1024-128 = 896.

So, if you have more than 1gb of RAM, it does not mean that your memory is not being used.

Anyhow, to get around this problem you could use a different PAGE_OFFSET or you could set HIGHMEM. HIGHMEM is known to consume extra cpu cycles, and use a little bit of memory itself. So from my experience, the best option if you have under 2gb of memory, is to edit the PAGE_OFFSET value. A 2gb:2gb split between user space and kernel space seems reasonable. Although this has obvious drawbacks since you're decreasing user space.

There are a lot more complexities to this whole issue, such as affecting the sbrk() heap and mmap() zones, and you should read about it more before deciding to either patch your kernel's PAGE_OFFSET value, or adding HIGHMEM support.

3

Unix / Linux Discussion / Re: Who has the most RAM ?

« on: September 27, 2005, 06:59:17 pm »

We should re-define the rules.

A linux box (used daily, none of this dual boot to occasionally use *nix as to keep ego up (LoRd)) at home (windowlicker) for the desktop (quik).

Heh, the top one was my personal linux box, and the output was from free. The bottom one, was a SunOS server from a research lab. SunOS lacks "free", so I used vmstat and mpstat.

4

Unix / Linux Discussion / Re: Who has the most RAM ?

« on: September 27, 2005, 05:47:57 pm »

free
                          total           used             free         shared         buffers        cached
Mem:       1035860    1011124        24736                    0       211448       281800
-/+ buffers/cache:         517876     517984
Swap:      2031608             160   2031448



I wish I had this sitting at home though.

vmstat
 procs     memory            page            disk          faults      cpu
 r b w   swap  free  re  mf pi po fr de sr f0 s0 s1 s2   in   sy   cs us sy id
 0 0 18  4480  1720   0   0  0  0  0  0  0  0  0 15  0 4294967196 0 0 -22 -7 -122

mpstat
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
  0   15   0  509   123  120   93    3    3    7    0   419   14   5   2  79
  1   15   0  512   172  169   92    3    4    8    0   397   14   5   2  80
  2   15   0  505   105  102   94    3    4    5    0   138   17   6   2  76
  3   15   0  196   448  246   90    3    4    5    0   373   14   4   1  81

5

General Programming / Re: Learning

« on: September 27, 2005, 05:19:30 pm »

If you want my advice on learning C, make sure to code perfectly. sometimes really sloppy and ugly code can result in memory leaks(mainly with pointers)

Sorry. Hate to break it to you, but if everything was coded "perfectly", we would have no use for patching software.

And by the way, "sloppy and ugly code" that results in "memory leaks" has nearly nothing to do with pointers. (Unless you push the wrong address space in the pointer integer, and point to something wrong, but that isn't a leak. That's more of an overflow than it is a leak.)

Do you know what a memory leak is? Apparently, seeing as how you associated it with pointers, I'd say no.

EDIT -- Did you know THAT good looking code can have memory leaks still?

Indeed. Most of my code never leaks memory, well according to Valgrind at least, but I'd be amazed if it ever passed a checkstyle. What might seem like ugly code to one programmer, could be perfectly acceptable to another. The international obfuscated C code contest comes to mind; I'd argue that the "ugly" source codes in those entries are pieces of art.

6

General Programming / Re: [ASM/C++] Function redirection

« on: September 26, 2005, 10:03:11 pm »

man ptrace

Looks like what I'll have to be using... bummer.

7

General Programming / Re: [ASM/C++] Function redirection

« on: September 26, 2005, 08:49:24 pm »

Are you doing this as a one-shot deal, or are you writing a program to do it? 

If you're writing a program, you can probably use WriteProcessMemory(). 

Program.... WriteProcessMemory isn't available on linux.

8

General Programming / Re: [ASM/C++] Function redirection

« on: September 26, 2005, 07:27:12 pm »

I don't plan on reverting to the original function.

Now to write to memory, would ptrace be the way to go?

9

General Programming / [ASM/C++] Function redirection

« on: September 26, 2005, 04:51:29 pm »

(gcc, nasm, Linux 2.6)

I'm in need of hijacking a non-virtual function in the Half Life 2 SDK. It is non-virtual, because obviously valve doesn't want to give developers access to it.
I've written a server plugin for the hl2 daemon, and this library contains the function which I want to use as a replacement for a standard function in one of the hl2 daemon shared libraries.

This is what I'm doing so far:
My plugin is loaded into the daemon's address space. The plugin finds the address of the function I want to replace using dlopen and dlsym.
The plugin creates a function pointer to the function which I want to use as a replacement.

How would I proceed in redirecting all calls to my function instead of the standard one? I'm not too keen on editing the .GOT or .PLT. I need the stack frame from the original function to stay in tact, because I will be using the parameters.

Any suggestions, with maybe some sample code?

Thanks in advance.