|
| ||
| Views: 1,609,369 | Main | Rules/FAQ | Memberlist | Active users | Last posts | Calendar | Stats | Online users | Search | 11-22-24 01:27 AM |
| Guest: | ||
| Main - Posts by d0k3 |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 1/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
I already asked something similar on GBAtemp, but haven't got much useful help there. What I'm trying to do is to copy a file efficiently on a small homebrew program I coded for Ninjhax (on N3DS, with CTRUlib and CTRcommon installed, if that matters). So far I've been using the C stdio.h functions (fopen / fread / fwrite), but it is awfully slow (around 100kB / sec, regardless of if directories are involved or not).
Here's my copy function: bool fsPathCopy(const std::string path, const std::string dest, std::function<bool(u64 pos, u64 totalSize)> onProgress) {
if(fsIsDirectory(path)) { if(mkdir(dest.c_str(), 0777) != 0) return false; std::vector<FileInfo> contents = fsGetDirectoryContents(path); for (std::vector<FileInfo>::iterator it = contents.begin(); it != contents.end(); it++) if (!fsPathCopy((*it).path, dest + "/" + (*it).name)) return false; return true; } else { bool ret = false; u8 buffer[BUFSIZ]; u64 total = fsGetFileSize(path); u64 pos = 0; u64 size; FILE* fp = fopen(path.c_str(), "rb"); FILE* fd = fopen(dest.c_str(), "wb"); if ((fp != NULL) && (fd != NULL)) { while ((size = fread(buffer, 1, BUFSIZ, fp)) > 0) { pos += fwrite(buffer, 1, size, fd); if((onProgress != NULL) && !onProgress(pos, total)) break; } ret = (pos == total); } if(fp != NULL) fclose(fp); if(fd != NULL) fclose(fd); return ret; } } I've seen that 4DsDev's own staplebutter prefers to use the file io function provided by CTRUlib, but their interfaces are somewhat clunky and I honestly wouldn't want to use them unless there is some kind of advantage in using them. So, is using the functions provided by CTRUlib instead of the standard C / CPP functions recommended? |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 2/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
I already tried increasing the buffer size to 1024 * 1024 byte - didn't help. The current buffer size, I don't know exactly BUFSIZ is defined by stdio.h. I think it's 8096 byte. So, I don't think the buffer size is the problem here... Are there any estimates on how fast reading from / writing to the internal SD should be (it's a 64GB UHS-1, btw)?
Installing .CIAs via FBI (which is a kind of copy operation, too) seems to work much faster, and the corresponding function uses a combination of fopen / fread (for reading the CIA) and functionality provided by CTRUlib (for installing / writing the CIA). However, the faster speed could be due to other reasons than using CTRUlib functions. Also: Thank you for your fast reply! |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 3/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
Alright, just an update. I fixed the slow speed issue (now it's comparable with the speed of other software out there), but how, I'm not exactly sure. I did increase the buffer size, and there where also some problems with my incorrect use of [&] lambda functions, so fixing up that my have solved that.
onProcess() itself had nothing to do with it, btw. While the problem still persisted, I disabed that with basically no effect on speed at all. Anyways, thank you! |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 4/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
I wondered if it is possible to access the NAND file system, and how to do it. The most comfortable way would be the standard functions (fopen and the not so standard opendir), of course.
CTRUlib seems to have some functionality that indicate it is possible, like FSUSER_GetNandArchiveResource, however using these seems awfully complicatedk, and I also have no idea what Smealum means by an archive. Any ideas? EDIT: I know, fiddling around with the NAND is dangerous, but the possibility to alter anything is not there if the standard functions work as expected. |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 5/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
Posted by yuriks Ah, exactly what I wanted to avoid  . So, I have to use the functions from fs.h for everything (that means not only "mount" the NAND, but also listing directories, opening files, etc...) and even then I won't have access to everything? |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 6/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
Posted by StapleButter Okay, thank you! Is there also no way with libkhax or Brahma (or similar libraries / tools)? On another note, I've learned what a devoptab is and found out CTRUlib has one for SD access, so I somehow hoped something similar would exist for NAND. |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 7/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
Posted by profi200 A buffer that large could possibly slow down things for smaller file operations... Still, good point. I'll try even bigger buffers then. |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 8/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
Posted by profi200 So, it should be possible via libkhax, correct? Posted by profi200 I can live without write access. All I want now is to be able to view these files. This is for my CTRXplorer project: https://github.com/d0k3/CTRXplorer Posted by profi200 Your interface actually looks pretty usable to me . But, could you specify these problems? Is read-only access stable and reliable enough? |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 9/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
Posted by StapleButter Oh well, I feel like a complete noob for asking stuff like this, but does that mean I actually have to change something in my ctrulib installation? Up till now I though just running khaxInit() would be enough. I use ctrcommon, which already includes libkhax. |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 10/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
Posted by profi200 Checked IO access flags mean we can access the data, correct? Only direct SDMC doesn't work (meaning we access SDMC through a layer). Correct? At least that's what I understood. Sorry, but I still have to learn some things
I guess I'll replace all my filesystem operations with your routines. That cannot hurt (vs fopen/fread/fwrite) anf it will open up a lot of possibilities. |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 11/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
Posted by profi200 So, if I can access anything besides the SD is unsure and (probably?) depends on the ARM9. I'll need to see if I find a solution to that, otherwise it doesn't make sense (now) to replace fopen/fread with something else. |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 12/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
Posted by profi200 I have a question about 'archives', as used by ctrulib and your interface... Are archives basically the same as FAT images? And, could, perhaps, the same functions also be used to read and write to an actual FAT image? |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 13/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
Posted by profi200 I have to make sure I got this right... o Your API and the ctrulib API cannot be used to read / write to FAT images on the SD card (btw. do you know a good alternative?) o However, I can access the raw, encrypted NAND via an archive IF I have special access. o Your API doesn't handle decrypting the data in this archive (I didn't see any crypto or xorpad stuff in there). Correct so far? |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 14/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
I'm currently working on a derivative of Decrypt9 and for that to load on my N3DS / FW 9.0.0 I need to use Brahma by patois:
https://github.com/patois/Brahma Everything works fine when I run Brahma first, then use the internal menu to run the Decrypt9 payload. However, I don't want to require myself / future users to go through the Brahma menu each and every time, so I just coded a small loader that loads the payload without user interaction. This is my code: #include <3ds.h>
#include "brahma.h" s32 main (void) { srvInit(); aptInit(); hidInit(NULL); gfxInitDefault(); fsInit(); sdmcInit(); hbInit(); qtmInit(); consoleInit(GFX_BOTTOM, NULL); if (brahma_init()) { load_arm9_payload(PAYLOAD); firm_reboot(); brahma_exit(); } hbExit(); sdmcExit(); fsExit(); gfxExit(); hidExit(); aptExit(); srvExit(); return 0; } Using Brahma as a library in the way above is intended by patois:
I'm also pretty sure half of the inits in the code above are completely useless, however, as I'm getting desperate, I'm trying to do it exactly the way patois does. In fact, save for a few hid input functions, this does exactly the same as Brahmas quick boot feature. I also tried to leave out part of the inits, but that lead to bad results (crashes). I haven't changed anything else in the Brahma source code. Now, if I use my loader to load the Decrypt9.bin payload, this is what I get: The above is actually using Shadowtrances bootstrap-mod code. The rectangles should be purple and the text should be green. The image above looks like some offset is wrong, but, again, it works fine if I go through untouched Brahma instead of using my loader. The problem also persists with other payloads, including patois' own example projects: https://github.com/patois/3DSProjects/tree/master/Brahma/ It's best seen with Shadowtrances Ui design, therefore I'm showing it as an example. Other functionality (besides screen output) seem to work. It even gets worse. I asked Shadowtrance to compile for me, assuming my ctrulib is too recent / something else is wrong in my toolchain. Download the binaries here if interested. Now, even with that, I got the same problems, while it works fine for Shadowtrance. My best bet so far is that it is something similar as with NTR CFW, where it fails more often for some users and waiting a few seconds helps. In fact, the OSKA project (which shares code with Brahma) has a delay function in their loader. I did try to implement a delay function in C, but to no avail so far. So... any ideas? My project is found here if you are interested. EDIT: Because this problem persists, as I wrote, even when following patois instructions and using only his code, I'd just open an issue on Github, but patois has disabled that option, and there's no way to get in touch other than a pull request (which I can't do, because I don't know any solution). |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 15/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
Yes, initially I didn't have the consoleInit() function in there, but I got desperate and untouched Brahma initializes the console before the exploit.
So, that function is not the reason behind the problem, it lies somewhere else. And, I'm actually only using the topscreen. However, the thing about RGB565 instead of RGB8, if I look at it now, somehow sounds plausible. I'll try and see if I can set the RGB mode manually before the exploit. |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 16/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
Alright, look at my fixed code:
#include <3ds.h>
#include "brahma.h" s32 main (void) { srvInit(); aptInit(); hidInit(NULL); gfxInitDefault(); fsInit(); sdmcInit(); hbInit(); qtmInit(); gfxSwapBuffers(); if (brahma_init()) { load_arm9_payload(PAYLOAD); firm_reboot(); brahma_exit(); } hbExit(); sdmcExit(); fsExit(); gfxExit(); hidExit(); aptExit(); srvExit(); return 0; } Meaning, if I run gfxSwapBuffers() once, the problem goes away for me. Also, patois is aware of the problem now as well. Thanks mid-kid! I guess on GBAtemp I would never have gotten near any solution for this problem. Now, the question remains as to how to make using Brahma as a library easier for future developers? I hope we'll find a good solution. |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 17/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
Posted by profi200 I just came here to ask exactly this, and there already is the answer. Thanks a million times! I'll try it and will let you know if I figured it out. |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 18/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
Posted by profi200 Okay, I guess I need some more help. For the CTR partitions, calculating the SHA256 hash and using the first 16 byte of that as CTR works, and is by far the superior method. However, I seem to can't get a usable result with the SHA1 hash for the TWL partitions. What I did to find out about the correct endianness is: * Generate a TWLN xorpad (with the SHA1 CTR, otherwise same as I would generate a CTRNAND xorpad. * Dump the encrypted TWLN partition. * Reorder the encrypted TWLN partition to change endianness - that means reversing the order of bytes on blocks of 2, 4, 8 or 16 bytes. Then use the xorpad, hoping to get a result with some recognizable data. So far, I had no success. Is anything wrong with my method? Maybe I even need new AES routines to be able to decrypt that? The new SHA1 hash algorithm is tested (it's from the same source as my SHA256 algorithms), and of course I'm always using a fresh copy of the encrypted TWLN before reordering. I'm using the offset from the start of the NAND (for the ctr_add function), same as I do for the CTRNAND. |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 19/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
Posted by profi200 Okay, now I'm pretty sure I got everything correctly. I used http://3dbrew.org/wiki/AES_Registers and doublechecked that with my crypto library "original version by megazig" (or here) . Keyslot is 0x3, too. Now, to even enable little endian / reversed order decryption, I had to change the code in the crypto library, from: void _decrypt(uint32_t value, void* inbuf, void* outbuf, size_t blocks)
to... (changes not on Github) (value has the bit set for AES CTR)
{ *REG_AESCNT = 0; *REG_AESBLKCNT = blocks << 16; *REG_AESCNT = value | AES_CNT_START | AES_CNT_INPUT_ORDER | AES_CNT_OUTPUT_ORDER | AES_CNT_INPUT_ENDIAN | AES_CNT_OUTPUT_ENDIAN | AES_CNT_FLUSH_READ | AES_CNT_FLUSH_WRITE; aes_fifos(inbuf, outbuf, blocks); } void _decrypt(uint32_t value, void* inbuf, void* outbuf, size_t blocks)
(I know, that breaks everything else, but it is just for testing)
{ *REG_AESCNT = 0; *REG_AESBLKCNT = blocks << 16; *REG_AESCNT = value | AES_CNT_START | AES_CNT_OUTPUT_ORDER | AES_CNT_OUTPUT_ENDIAN | AES_CNT_FLUSH_READ | AES_CNT_FLUSH_WRITE; aes_fifos(inbuf, outbuf, blocks); } The setctr function handles everything correctly (if I give the correct order / endian modes). No change needed: void set_ctr(int mode, void* iv)
{ uint32_t * _iv = (uint32_t*)iv; *REG_AESCNT = (*REG_AESCNT & ~(AES_CNT_INPUT_ENDIAN|AES_CNT_INPUT_ORDER)) | (mode << 23); if (mode & AES_NORMAL_INPUT) { *(REG_AESCTR + 0) = _iv[3]; *(REG_AESCTR + 1) = _iv[2]; *(REG_AESCTR + 2) = _iv[1]; *(REG_AESCTR + 3) = _iv[0]; } else { *(REG_AESCTR + 0) = _iv[0]; *(REG_AESCTR + 1) = _iv[1]; *(REG_AESCTR + 2) = _iv[2]; *(REG_AESCTR + 3) = _iv[3]; } } Now, the crypto.h / crypto.c library looks like it has been tampered with (or edited by too many people), even before I made my additional change. I guess it should still be usable. As I said, decrypting the CTRNAND works with the same routines. For TWL, I calculated the CTR via SHA-1 (only the first 16 bytes), changed the keyslot, changed the set mode to reversed and little endian (same for decryption mode, see above). Other than that, everything stays the same. I also tried incrementing the counter in a different way once, but that doesn't help either. Usable output should be pretty easy to recognize as I have the offsets for the start of the FAT16 / FAT12 images. In other words, the first 16 bytes are enough to decide if everything is okay (I looked further, of course). So, why I'm writing all this? It still doesn't give usable output :|. Even with my method from before (this time generating xorpads correctly), I do not get anything useful. Do you have any additional ideas? Maybe there is some interim result that I could check somewhere, so that I can be sure my implementation is correct at least to that point? And did someone already manage to decrypt the TWLNAND without resorting to archives? And maybe there's some place on the web where there's mroe information about TWL decryption (didn't find anything usable on 3Dbrew). EDIT: Maybe there are prerequisites to being able to decrypt the TWL? http://3dbrew.org/wiki/AES_Registers#Keyslots I stumbled over the 'NATIVE_FIRM hard-boot' in that table, and 'probably unset' in table right above. If that is useful information, I'm using Brahma as a loader. EDIT2: Okay, I dug a little deeper and found this: http://dsibrew.org/wiki/Bootloader
Could that mean I need to retrieve and set the KeyY? (that wasn't needed for CTRNAND decyption) Sorry for the long post, too. |
| d0k3 |
| ||
|
Member Normal user Level: 20 Posts: 20/75 EXP: 38197 Next: 4242 Since: 06-04-15 Last post: 3251 days ago Last view: 2998 days ago |
Posted by profi200 Alright, thanks a ton, again. From the first look I think my error was that I did not manually change the endianness ("EndianSwap") of the CTR (I did, in some of my experiments, but in those I did other things wrong). I'll have to thoroughly look it through. BTW: I already made a local copy, just in case that gist was only temporary. Update: I got it. What I needed to do was to manually reverse the word order and endian (in fact, reversing it byte by byte) for the SHA-1 hash before doing anything else. Then increment it and start decoding. If I do it any other way (ie. use the AES parameters to handle endian in setctr), the add ctr function doesn't work in the correct way. So, what I got wrong was the incrementation - I thought that should work the same, regardless of endian / order. I also had to change word order / endian (via the respective parameters, not for the data) for the actual decryption, but that I had before. Thank you again! The set AES bits 12 and 13, btw, don't seem to do anything, and they are undocumented on 3dbrew. |
| Main - Posts by d0k3 |
|
Page rendered in 0.046 seconds. (2048KB of memory used) MySQL - queries: 22, rows: 99/99, time: 0.005 seconds. ![[powered by Acmlm]](img/poweredbyacmlm.png "powering nostalgia")
Acmlmboard 2.064 (2018-07-20)© 2005-2008 Acmlm, Xkeeper, blackhole89 et al. |