4dsdev
Views: 614,262 Main | Rules/FAQ | Memberlist | Active users | Last posts | Calendar | Stats | Online users | Search 11-25-17 06:33 AM
Guest:

Main - Posts by d0k3

Pages: 1 2 3 4
d0k3
Posted on 06-04-15 08:54 AM, in Copying a file efficiently (on Ninjhax / N3DS) Link | #170
I already asked something similar on GBAtemp, but haven't got much useful help there. What I'm trying to do is to copy a file efficiently on a small homebrew program I coded for Ninjhax (on N3DS, with CTRUlib and CTRcommon installed, if that matters). So far I've been using the C stdio.h functions (fopen / fread / fwrite), but it is awfully slow (around 100kB / sec, regardless of if directories are involved or not).

Here's my copy function:
bool fsPathCopy(const std::string path, const std::string dest, std::function<bool(u64 pos, u64 totalSize)> onProgress) {
if(fsIsDirectory(path)) {
if(mkdir(dest.c_str(), 0777) != 0) return false;
std::vector<FileInfo> contents = fsGetDirectoryContents(path);
for (std::vector<FileInfo>::iterator it = contents.begin(); it != contents.end(); it++)
if (!fsPathCopy((*it).path, dest + "/" + (*it).name)) return false;
return true;
} else {
bool ret = false;
u8 buffer[BUFSIZ];
u64 total = fsGetFileSize(path);
u64 pos = 0;
u64 size;
FILE* fp = fopen(path.c_str(), "rb");
FILE* fd = fopen(dest.c_str(), "wb");
if ((fp != NULL) && (fd != NULL)) {
while ((size = fread(buffer, 1, BUFSIZ, fp)) > 0) {
pos += fwrite(buffer, 1, size, fd);
if((onProgress != NULL) && !onProgress(pos, total)) break;
}
ret = (pos == total);
}
if(fp != NULL) fclose(fp);
if(fd != NULL) fclose(fd);
return ret;
}
}

I've seen that 4DsDev's own staplebutter prefers to use the file io function provided by CTRUlib, but their interfaces are somewhat clunky and I honestly wouldn't want to use them unless there is some kind of advantage in using them. So, is using the functions provided by CTRUlib instead of the standard C / CPP functions recommended?

d0k3
Posted on 06-04-15 09:51 AM, in Copying a file efficiently (on Ninjhax / N3DS) (rev. 3 of 06-04-15 09:53 AM) Link | #172
I already tried increasing the buffer size to 1024 * 1024 byte - didn't help. The current buffer size, I don't know exactly BUFSIZ is defined by stdio.h. I think it's 8096 byte. So, I don't think the buffer size is the problem here... Are there any estimates on how fast reading from / writing to the internal SD should be (it's a 64GB UHS-1, btw)?

Installing .CIAs via FBI (which is a kind of copy operation, too) seems to work much faster, and the corresponding function uses a combination of fopen / fread (for reading the CIA) and functionality provided by CTRUlib (for installing / writing the CIA). However, the faster speed could be due to other reasons than using CTRUlib functions.

Also: Thank you for your fast reply!

d0k3
Posted on 06-07-15 08:03 PM, in Copying a file efficiently (on Ninjhax / N3DS) Link | #182
Alright, just an update. I fixed the slow speed issue (now it's comparable with the speed of other software out there), but how, I'm not exactly sure. I did increase the buffer size, and there where also some problems with my incorrect use of [&] lambda functions, so fixing up that my have solved that.

onProcess() itself had nothing to do with it, btw. While the problem still persisted, I disabed that with basically no effect on speed at all.

Anyways, thank you!


d0k3
Posted on 06-07-15 08:10 PM, in Accessing the NAND (via fopen / opendir) (rev. 2 of 06-07-15 08:13 PM) Link | #184
I wondered if it is possible to access the NAND file system, and how to do it. The most comfortable way would be the standard functions (fopen and the not so standard opendir), of course.

CTRUlib seems to have some functionality that indicate it is possible, like FSUSER_GetNandArchiveResource, however using these seems awfully complicatedk, and I also have no idea what Smealum means by an archive.

Any ideas?


EDIT: I know, fiddling around with the NAND is dangerous, but the possibility to alter anything is not there if the standard functions work as expected.

d0k3
Posted on 06-08-15 03:29 AM, in Accessing the NAND (via fopen / opendir) Link | #187
Posted by yuriks
As far as I know applications don't have direct access to the NAND filesystem. You can access restricted subsets of data (mostly shared data offered by other applications in the system) but that will require using the FS service functions: http://3dbrew.org/wiki/FS:OpenArchive

Ah, exactly what I wanted to avoid :). So, I have to use the functions from fs.h for everything (that means not only "mount" the NAND, but also listing directories, opening files, etc...) and even then I won't have access to everything?

d0k3
Posted on 06-08-15 10:16 AM, in Accessing the NAND (via fopen / opendir) Link | #189
Posted by StapleButter
As yuriks said, you can't access everything. You can't just use FS_OpenArchive with the NAND archive IDs. Regular apps don't have access to that.

You would need to build a .cia (or .3ds) with a proper .rsf specifying NAND access.

Okay, thank you! Is there also no way with libkhax or Brahma (or similar libraries / tools)?

On another note, I've learned what a devoptab is and found out CTRUlib has one for SD access, so I somehow hoped something similar would exist for NAND.

d0k3
Posted on 06-09-15 12:25 PM, in Copying a file efficiently (on Ninjhax / N3DS) Link | #192
Posted by profi200
Nintendos API is fucking slow. It DMAs the buffer 2 times before it is in your apps heap. I myself would not go below 2 MB of buffer size.


A buffer that large could possibly slow down things for smaller file operations... Still, good point. I'll try even bigger buffers then.

d0k3
Posted on 06-09-15 12:30 PM, in Accessing the NAND (via fopen / opendir) Link | #193
Posted by profi200
You can only access nandrw and nandro on ARM11 through archives.
See http://3dbrew.org/wiki/Flash_Filesystem and http://3dbrew.org/wiki/FS:OpenArchive

So, it should be possible via libkhax, correct?

Posted by profi200
There is a way to get rw access to nandro apparently but i never got that working. Probably FSPXI-only.

I can live without write access. All I want now is to be able to view these files. This is for my CTRXplorer project:
https://github.com/d0k3/CTRXplorer

Posted by profi200
Besides that the problems outweigh the benefits. If you want to implement it nevertheless you also maybe want to take a look at my fs interface which simplyfies using Nintendos API: https://github.com/profi200/sysUpdater/blob/master/include/fs.h (There is a bug however i just recently discovered. Using "/" as path doesn't work for copyDir()). It uses UTF-16 paths but that's not a big problem.

Your interface actually looks pretty usable to me :). But, could you specify these problems? Is read-only access stable and reliable enough?

d0k3
Posted on 06-09-15 02:35 PM, in Accessing the NAND (via fopen / opendir) Link | #197
Posted by StapleButter
Yeah, although you'll need to patch the FS module to remove access restrictions.

Oh well, I feel like a complete noob for asking stuff like this, but does that mean I actually have to change something in my ctrulib installation? Up till now I though just running khaxInit() would be enough. I use ctrcommon, which already includes libkhax.

d0k3
Posted on 06-10-15 04:55 PM, in Accessing the NAND (via fopen / opendir) Link | #201
Posted by profi200
Other IO access flags are checked it seems. Only the direct SDMC one is not. No idea if the ARM9 does this.


Checked IO access flags mean we can access the data, correct? Only direct SDMC doesn't work (meaning we access SDMC through a layer). Correct? At least that's what I understood. Sorry, but I still have to learn some things :)

I guess I'll replace all my filesystem operations with your routines. That cannot hurt (vs fopen/fread/fwrite) anf it will open up a lot of possibilities.


d0k3
Posted on 06-12-15 05:48 AM, in Accessing the NAND (via fopen / opendir) Link | #207
Posted by profi200
It's the other way around. You can always get SD card access somehow but if the other flags are checked on ARM9 you can do nothing about it from the ARM11 side alone.

So, if I can access anything besides the SD is unsure and (probably?) depends on the ARM9. I'll need to see if I find a solution to that, otherwise it doesn't make sense (now) to replace fopen/fread with something else.

d0k3
Posted on 06-17-15 04:01 PM, in Accessing the NAND (via fopen / opendir) Link | #215
Posted by profi200
Try it out. No idea if libhax patches the permissions on ARM11. But if the ARM9 checks them too that will not work.

I guess i will change the exception stuff again slightly. I have just recently started using exceptions.

I have a question about 'archives', as used by ctrulib and your interface... Are archives basically the same as FAT images? And, could, perhaps, the same functions also be used to read and write to an actual FAT image?

d0k3
Posted on 06-18-15 08:23 AM, in Accessing the NAND (via fopen / opendir) Link | #218
Posted by profi200
Archives are objects with informations for Nintendos API which does all in background.
Archives for RAW unencrypted partition access don't exist. It's all encapsulated in their API. However an archive for RAW NAND access exists (requires special access like the other archives). To get that working you need ARM9 access or xorpads to en-/decrypt on the fly.


I have to make sure I got this right...
o Your API and the ctrulib API cannot be used to read / write to FAT images on the SD card (btw. do you know a good alternative?)
o However, I can access the raw, encrypted NAND via an archive IF I have special access.
o Your API doesn't handle decrypting the data in this archive (I didn't see any crypto or xorpad stuff in there).
Correct so far?

d0k3
Posted on 06-22-15 07:41 AM, in Using Brahma as a library / loader (rev. 2 of 06-22-15 10:50 AM) Link | #228
I'm currently working on a derivative of Decrypt9 and for that to load on my N3DS / FW 9.0.0 I need to use Brahma by patois:
https://github.com/patois/Brahma

Everything works fine when I run Brahma first, then use the internal menu to run the Decrypt9 payload. However, I don't want to require myself / future users to go through the Brahma menu each and every time, so I just coded a small loader that loads the payload without user interaction.

This is my code:
#include <3ds.h>
#include "brahma.h"

s32 main (void) {
srvInit();
aptInit();
hidInit(NULL);
gfxInitDefault();
fsInit();
sdmcInit();
hbInit();
qtmInit();

consoleInit(GFX_BOTTOM, NULL);
if (brahma_init()) {
load_arm9_payload(PAYLOAD);
firm_reboot();
brahma_exit();
}

hbExit();
sdmcExit();
fsExit();
gfxExit();
hidExit();
aptExit();
srvExit();

return 0;
}

Using Brahma as a library in the way above is intended by patois:
Brahma and its exploits which enable privileged ARM9 code execution
on the Nintendo 3DS may also be used as a "library" (#include "brahma.h")

- call brahma_init() - initializes Brahma
- call load_arm9_payload() - loads a payload binary from disk
- call firm_reboot() - executes a payload binary (privileged ARM9 code)
- (please check the source code for more features and options)

I'm also pretty sure half of the inits in the code above are completely useless, however, as I'm getting desperate, I'm trying to do it exactly the way patois does. In fact, save for a few hid input functions, this does exactly the same as Brahmas quick boot feature. I also tried to leave out part of the inits, but that lead to bad results (crashes). I haven't changed anything else in the Brahma source code.

Now, if I use my loader to load the Decrypt9.bin payload, this is what I get:
[image]
The above is actually using Shadowtrances bootstrap-mod code. The rectangles should be purple and the text should be green. The image above looks like some offset is wrong, but, again, it works fine if I go through untouched Brahma instead of using my loader. The problem also persists with other payloads, including patois' own example projects:
https://github.com/patois/3DSProjects/tree/master/Brahma/
It's best seen with Shadowtrances Ui design, therefore I'm showing it as an example. Other functionality (besides screen output) seem to work.

It even gets worse. I asked Shadowtrance to compile for me, assuming my ctrulib is too recent / something else is wrong in my toolchain. Download the binaries here if interested. Now, even with that, I got the same problems, while it works fine for Shadowtrance.

My best bet so far is that it is something similar as with NTR CFW, where it fails more often for some users and waiting a few seconds helps. In fact, the OSKA project (which shares code with Brahma) has a delay function in their loader. I did try to implement a delay function in C, but to no avail so far.


So... any ideas? My project is found here if you are interested.


EDIT: Because this problem persists, as I wrote, even when following patois instructions and using only his code, I'd just open an issue on Github, but patois has disabled that option, and there's no way to get in touch other than a pull request (which I can't do, because I don't know any solution).

d0k3
Posted on 06-22-15 03:19 PM, in Using Brahma as a library / loader (rev. 3 of 06-22-15 03:27 PM) Link | #232
Yes, initially I didn't have the consoleInit() function in there, but I got desperate and untouched Brahma initializes the console before the exploit.

So, that function is not the reason behind the problem, it lies somewhere else. And, I'm actually only using the topscreen.

However, the thing about RGB565 instead of RGB8, if I look at it now, somehow sounds plausible. I'll try and see if I can set the RGB mode manually before the exploit.

d0k3
Posted on 06-22-15 06:31 PM, in Using Brahma as a library / loader (rev. 2 of 06-22-15 06:32 PM) Link | #234
Alright, look at my fixed code:
#include <3ds.h>
#include "brahma.h"

s32 main (void) {
srvInit();
aptInit();
hidInit(NULL);
gfxInitDefault();
fsInit();
sdmcInit();
hbInit();
qtmInit();

gfxSwapBuffers();
if (brahma_init()) {
load_arm9_payload(PAYLOAD);
firm_reboot();
brahma_exit();
}

hbExit();
sdmcExit();
fsExit();
gfxExit();
hidExit();
aptExit();
srvExit();

return 0;
}

Meaning, if I run gfxSwapBuffers() once, the problem goes away for me. Also, patois is aware of the problem now as well. Thanks mid-kid! I guess on GBAtemp I would never have gotten near any solution for this problem.

Now, the question remains as to how to make using Brahma as a library easier for future developers? I hope we'll find a good solution.

d0k3
Posted on 07-04-15 01:05 PM, in Accessing the NAND (via fopen / opendir) Link | #245
Posted by profi200
I saw your posts on GBAfail.

The method people get the correct CTR is unnecessary complicated. For CTR related partitions the CTR is the first half of a SHA256 hash over the NAND CID which can be found at the same place every time (see http://3dbrew.org/wiki/Memory_layout#ARM9_ITCM). For TWL partitions it's the first 16 bytes of a SHA1 hash over the NAND CID. The endianess/word order is different for TWL partitions so you need to experiment a bit with it. Otherwise it works exactly the same way as for CTR partitions.

Sorry for the double post. I think otherwise no one notices it :p

I just came here to ask exactly this, and there already is the answer. Thanks a million times! I'll try it and will let you know if I figured it out.

d0k3
Posted on 07-04-15 09:54 PM, in Accessing the NAND (via fopen / opendir) (rev. 5 of 07-05-15 08:29 PM) Link | #246
Posted by profi200
I saw your posts on GBAfail.

The method people get the correct CTR is unnecessary complicated. For CTR related partitions the CTR is the first half of a SHA256 hash over the NAND CID which can be found at the same place every time (see http://3dbrew.org/wiki/Memory_layout#ARM9_ITCM). For TWL partitions it's the first 16 bytes of a SHA1 hash over the NAND CID. The endianess/word order is different for TWL partitions so you need to experiment a bit with it. Otherwise it works exactly the same way as for CTR partitions.


Okay, I guess I need some more help. For the CTR partitions, calculating the SHA256 hash and using the first 16 byte of that as CTR works, and is by far the superior method.

However, I seem to can't get a usable result with the SHA1 hash for the TWL partitions. What I did to find out about the correct endianness is:
* Generate a TWLN xorpad (with the SHA1 CTR, otherwise same as I would generate a CTRNAND xorpad.
* Dump the encrypted TWLN partition.
* Reorder the encrypted TWLN partition to change endianness - that means reversing the order of bytes on blocks of 2, 4, 8 or 16 bytes. Then use the xorpad, hoping to get a result with some recognizable data.

So far, I had no success. Is anything wrong with my method? Maybe I even need new AES routines to be able to decrypt that? The new SHA1 hash algorithm is tested (it's from the same source as my SHA256 algorithms), and of course I'm always using a fresh copy of the encrypted TWLN before reordering. I'm using the offset from the start of the NAND (for the ctr_add function), same as I do for the CTRNAND.

d0k3
Posted on 07-06-15 05:30 PM, in Accessing the NAND (via fopen / opendir) (rev. 9 of 07-07-15 06:20 AM) Link | #248
Posted by profi200
- Keyslot is 3 for both twln and p.
- CTR is the first 16 bytes of the SHA1 hash. Needs to be set with reversed word order and little endian.
- The en-/decryption params are little endian and reversed word order too.

Okay, now I'm pretty sure I got everything correctly. I used http://3dbrew.org/wiki/AES_Registers and doublechecked that with my crypto library "original version by megazig" (or here) . Keyslot is 0x3, too.

Now, to even enable little endian / reversed order decryption, I had to change the code in the crypto library, from:
void _decrypt(uint32_t value, void* inbuf, void* outbuf, size_t blocks)
{
*REG_AESCNT = 0;
*REG_AESBLKCNT = blocks << 16;
*REG_AESCNT = value |
AES_CNT_START |
AES_CNT_INPUT_ORDER |
AES_CNT_OUTPUT_ORDER |
AES_CNT_INPUT_ENDIAN |
AES_CNT_OUTPUT_ENDIAN |
AES_CNT_FLUSH_READ |
AES_CNT_FLUSH_WRITE;
aes_fifos(inbuf, outbuf, blocks);
}
to... (changes not on Github) (value has the bit set for AES CTR)
void _decrypt(uint32_t value, void* inbuf, void* outbuf, size_t blocks)
{
*REG_AESCNT = 0;
*REG_AESBLKCNT = blocks << 16;
*REG_AESCNT = value |
AES_CNT_START |
AES_CNT_OUTPUT_ORDER |
AES_CNT_OUTPUT_ENDIAN |
AES_CNT_FLUSH_READ |
AES_CNT_FLUSH_WRITE;
aes_fifos(inbuf, outbuf, blocks);
}
(I know, that breaks everything else, but it is just for testing)

The setctr function handles everything correctly (if I give the correct order / endian modes). No change needed:
void set_ctr(int mode, void* iv)
{
uint32_t * _iv = (uint32_t*)iv;
*REG_AESCNT = (*REG_AESCNT & ~(AES_CNT_INPUT_ENDIAN|AES_CNT_INPUT_ORDER)) | (mode << 23);
if (mode & AES_NORMAL_INPUT)
{
*(REG_AESCTR + 0) = _iv[3];
*(REG_AESCTR + 1) = _iv[2];
*(REG_AESCTR + 2) = _iv[1];
*(REG_AESCTR + 3) = _iv[0];
}
else
{
*(REG_AESCTR + 0) = _iv[0];
*(REG_AESCTR + 1) = _iv[1];
*(REG_AESCTR + 2) = _iv[2];
*(REG_AESCTR + 3) = _iv[3];
}
}

Now, the crypto.h / crypto.c library looks like it has been tampered with (or edited by too many people), even before I made my additional change. I guess it should still be usable.

As I said, decrypting the CTRNAND works with the same routines. For TWL, I calculated the CTR via SHA-1 (only the first 16 bytes), changed the keyslot, changed the set mode to reversed and little endian (same for decryption mode, see above). Other than that, everything stays the same. I also tried incrementing the counter in a different way once, but that doesn't help either.

Usable output should be pretty easy to recognize as I have the offsets for the start of the FAT16 / FAT12 images. In other words, the first 16 bytes are enough to decide if everything is okay (I looked further, of course).

So, why I'm writing all this? It still doesn't give usable output :|. Even with my method from before (this time generating xorpads correctly), I do not get anything useful.

Do you have any additional ideas? Maybe there is some interim result that I could check somewhere, so that I can be sure my implementation is correct at least to that point? And did someone already manage to decrypt the TWLNAND without resorting to archives? And maybe there's some place on the web where there's mroe information about TWL decryption (didn't find anything usable on 3Dbrew).


EDIT: Maybe there are prerequisites to being able to decrypt the TWL?
http://3dbrew.org/wiki/AES_Registers#Keyslots
I stumbled over the 'NATIVE_FIRM hard-boot' in that table, and 'probably unset' in table right above. If that is useful information, I'm using Brahma as a loader.

EDIT2: Okay, I dug a little deeper and found this:
http://dsibrew.org/wiki/Bootloader
Stage1 uses the AES_Engine with mode AES-CTR to decrypt each ARM9/ARM7 binary, where keyY is from the above signature. The AES_Engine keyslot used here is the same one used for the shared areas for Tad, therefore the keyX is the same as the one used for that. The following is used for the CTR, where "binblk->binblocksize" is the above binary size aligned to 0x200-bytes:

Could that mean I need to retrieve and set the KeyY? (that wasn't needed for CTRNAND decyption)

Sorry for the long post, too.

d0k3
Posted on 07-07-15 03:30 PM, in Accessing the NAND (via fopen / opendir) (rev. 5 of 07-08-15 03:14 AM) Link | #250
Posted by profi200
No.

https://gist.github.com/profi200/469f9fac6a394d23e8e1
Very old code but it worked. I just slapped some comments in it. The way i calculated the CTR with the offset is really shit and could overflow. For me it worked.

Alright, thanks a ton, again. From the first look I think my error was that I did not manually change the endianness ("EndianSwap") of the CTR (I did, in some of my experiments, but in those I did other things wrong). I'll have to thoroughly look it through.

BTW: I already made a local copy, just in case that gist was only temporary.

Update: I got it. What I needed to do was to manually reverse the word order and endian (in fact, reversing it byte by byte) for the SHA-1 hash before doing anything else. Then increment it and start decoding. If I do it any other way (ie. use the AES parameters to handle endian in setctr), the add ctr function doesn't work in the correct way. So, what I got wrong was the incrementation - I thought that should work the same, regardless of endian / order. I also had to change word order / endian (via the respective parameters, not for the data) for the actual decryption, but that I had before. Thank you again!

The set AES bits 12 and 13, btw, don't seem to do anything, and they are undocumented on 3dbrew.
Pages: 1 2 3 4

Main - Posts by d0k3

Page rendered in 0.066 seconds. (2048KB of memory used)
MySQL - queries: 22, rows: 99/99, time: 0.030 seconds.
[powered by Acmlm] Acmlmboard 2.064 (2017-11-20)
© 2005-2008 Acmlm, Xkeeper, blackhole89 et al.