|
| ||
| Views: 1,393,847 | Main | Rules/FAQ | Memberlist | Active users | Last posts | Calendar | Stats | Online users | Search | 04-19-24 01:40 AM |
| Guest: | ||
| Main - Posts by neobrain |
| neobrain |
| ||
|
Member Normal user Level: 10 Posts: 1/17 EXP: 4112 Next: 302 Since: 11-15-14 Last post: 3097 days ago Last view: 2586 days ago |
WHAT IS NIHSTRO?
nihstro is a collection of tools for dealing with 3DS shader developement and reverse engineering. Currently it accommodates an assembler and a disassembler. The assembly shading language used by nihstro follows a public specification. Included with nihstro is a description of the shader assembly language and a concrete list instruction set reference. Additionally, two very simple example applications are included to illustrate usage (shaders are bundled in the data subdirectories). nihstro is written in C++11 and requires the Boost libraries. You can read about the details on the GitHub page. BINARIES Windows binaries can be downloaded REPORTING ASSEMBLER BUGS Given its early development state, there may be a number of quirks with the assembler, in particular: * Code which you would intuitively expect to work might not compile, or compile in an unexpected way * Statements which are valid according to the specification or the instruction set reference may be rejected as invalid, or generate an incorrect shader binary. If you encounter any of these issues or have any other suggestions to make, please report them in this thread (and include an example shader source and the output shbin in that report). You can also create an issue on GitHub. |
| neobrain |
| ||
|
Member Normal user Level: 10 Posts: 2/17 EXP: 4112 Next: 302 Since: 11-15-14 Last post: 3097 days ago Last view: 2586 days ago |
I mentioned this on IRC already, but for reference: I think talking about this in a shader language just adds a second level of confusion, hence I suggest providing the underlying bytecode in hex along to each assembly line. That's probably a bit more effort, but at least we then can be sure that if something "odd" is happening it is because of the GPU, rather than due to some misdesign of the shader assembler.
For example, I think one issue with what you explained out is that you try to assign a floating point register to the second source argument of the mul instruction. However, as far as I am aware src2 only has 5 bits available for addressing registers, which is why 0x25 cannot be indexed by that (contrary to src1, which has 7 bits). Obviously, this issue only manifests itself in aemstro's shading language, because had you worked with bytecode directly, you wouldn't even have thought of trying to assign 0x25 to src2. By the way, nihstro is aware of this limitation and hence swaps the values of src1 and src2 if necessary. |
| neobrain |
| ||
|
Member Normal user Level: 10 Posts: 3/17 EXP: 4112 Next: 302 Since: 11-15-14 Last post: 3097 days ago Last view: 2586 days ago |
Just gave this a quick try - seems to work fine on my 3DS  |
| neobrain |
| ||
|
Member Normal user Level: 10 Posts: 4/17 EXP: 4112 Next: 302 Since: 11-15-14 Last post: 3097 days ago Last view: 2586 days ago |
Fwiw, I just uploaded a Windows binary (link in the first post). |
| neobrain |
| ||
|
Member Normal user Level: 10 Posts: 5/17 EXP: 4112 Next: 302 Since: 11-15-14 Last post: 3097 days ago Last view: 2586 days ago |
Nice findings so far
If you haven't already, you should definitely read the publications "A Real-Time Configurable Shader Based on Lookup Tables" (Ohbuchi & Unno) and "Shading by Quaternion Interpolation" (Hast). Those two papers basically give away the whole algorithm used in hardware, and will give you a hint of what to look for. |
| neobrain |
| ||
|
Member Normal user Level: 10 Posts: 6/17 EXP: 4112 Next: 302 Since: 11-15-14 Last post: 3097 days ago Last view: 2586 days ago |
Just a quick overview:
smea's aemstro: - scope: assembler + disassembler - dependencies: python3 (runtime) - history: was the first shader (dis)assembler to be published and kind of a playground for reverse-engineering shaders - usability: it works, but isn't really tailored for easy or convenient use. - prospects: I think given the alternatives and smea's amount of free time, there are no plans on making it more usable for homebrew developers. neobrain's nihstro: - scope: assembler + disassembler + C++-library - dependencies: boost (compile-time), C++11 (compile-time), CMake (compile-time) - history: created because a shader disassembler had to be written for Citra anyway; the assembler component is the result of me wanting to try out boost::spirit. - usability: shader language is design for convenient usage - prospects: actively maintained, but the assembler hasn't received any work recently, mostly because of the seemingly general lack of interest in GPU homebrew development. fincs's picasso: - scope: assembler - dependencies: autotools, TDM-GCC (on Windows; compile-time) - history: written after aemstro and nihstro because both were considered to be lacking in different aspects by the author - usability: follows syntax similar to nihstro - prospects: it hasn't been updated in a while. It's only an assembler. My (biased) opinion is that aemstro isn't going anywhere and picasso is built on arguably inferior technology, but probably "good enough" for most people. As for nihstro, I would probably put the assembler portion higher on my TODO list if more people actually started creating 3DS homebrew using the GPU. In any case, picasso is probably the only thing suitable for inclusion into devkitARM, because including nihstro (apparently) means including boost, which is (understandably) unacceptable for the DKA maintainers. None of that is relevant for homebrew developers though, since those will probably use a binary distribution anyway. However, I don't think we desperately need to have a shader assembler included in DKA if that makes for a less pleasant development experience, especially since we don't really know yet which of the three projects will remain active in the long run. If people insist on shipping *something* with DKA, I would say let things cool down for another few months and see which of the three projects is most advanced by then and/or most commonly used by homebrew developers. That's better than imposing some default onto people (who will probably think it's the only option available) and making them convert over half their code base when they realize the external tools end up being superior. If nihstro turns out to be the favorite, then I guess it would be best to simply not ever include a shader assembler in DKA. |
| neobrain |
| ||
|
Member Normal user Level: 10 Posts: 7/17 EXP: 4112 Next: 302 Since: 11-15-14 Last post: 3097 days ago Last view: 2586 days ago |
Just put out an official 0.1 pre-release, link in the first post!
Quoting announcement from gbatemp:
|
| neobrain |
| ||
|
Member Normal user Level: 10 Posts: 8/17 EXP: 4112 Next: 302 Since: 11-15-14 Last post: 3097 days ago Last view: 2586 days ago |
Posted by Tcm0 Unfortunately I don't have too much time on my hands to come up with new examples, so any contributions would be welcome there. As for texturing triangles, the cube demo will show you how to do that already. Displaying backgrounds might be as simple as rendering a textured quad (= two triangles) spanning the whole screen; something more sophisticated is known as "sky boxes", and should be easy enough to find using Google. |
| neobrain |
| ||
|
Member Normal user Level: 10 Posts: 9/17 EXP: 4112 Next: 302 Since: 11-15-14 Last post: 3097 days ago Last view: 2586 days ago |
And finally, the annoying "new" icon on the "Unbricking and fixing" section is gone  |
| neobrain |
| ||
|
Member Normal user Level: 10 Posts: 10/17 EXP: 4112 Next: 302 Since: 11-15-14 Last post: 3097 days ago Last view: 2586 days ago |
Normmatt found some more info and uploaded it to https://gist.github.com/Normmatt/9893b46b1bca86af3fd4#file-pica_fragment_light . In particular, this section looks interesting:
Posted by that link This seems to confirm your findings in your post about LUTs. According to DMP's research papers, SP models a spotlight if the proper lookup function (namely -L*P) is used, so it should probably be renamed to "spotlight" indeed. Some further information on the LUTs: They take a one-dimensional index, which is user-configurable. The index is calculated from the set (N*H, V*H, L*N, -L*P, cos Phi), with P the spotlight direction, Phi "the angle between the projection of H onto the tangent plane and the tangent vector T". I suppose the LUT index is set up by PICA_REG_FRAG_LIGHT_LUTINPUT, i.e. GPU register 0x1D1 (this is pure speculation based on that register's name, though). Some of the vectors in these formulas are likely configured via GPU registers, while others are vertex shader outputs. In particular, the view vector V is given by the "view" output vertex attribute semantic, while the normal vector N and tangent vector T are calculated from the "quaternion" output vertex attribute semantic. H should simply be something like (V+N)/2 (or its normalization). The precise calculation of N and T happens by using the quaternion to transform the surface-local space into eyespace (which is really just a simple rotation, and hence can indeed be encoded in a unit quaternion). In surface-local space, N and T always take fixed values (e.g. N=(0,0,1) and T=(0,1,0)). Hence all required information about the eye-space values really is encoded in the quaternion. Side note: Quaternion-vector multiplication is a linear operation, hence to get per-fragment values of N and T it should be sufficient to linearly interpolate the quaternion. |
| neobrain |
| ||
|
Member Normal user Level: 10 Posts: 11/17 EXP: 4112 Next: 302 Since: 11-15-14 Last post: 3097 days ago Last view: 2586 days ago |
New Windows x86 binaries: revision c3d7977888cd New builds below. |
| neobrain |
| ||
|
Member Normal user Level: 10 Posts: 12/17 EXP: 4112 Next: 302 Since: 11-15-14 Last post: 3097 days ago Last view: 2586 days ago |
Ever wanted to backup your 3DS games for usage with Citra, but didn't feel like resorting to piracy-oriented solutions like Gateway? Looking for a less painful way to decrypt game dumps, which doesn't require a gazillion of different tools? Wait no longer, here's uncart!
uncart is a tool for dumping 3DS game cartridges that I have been working on with a number of other people over the past few weeks. It will dump any 3DS game cartridge to the SD card, from which you can transfer it to your computer (by just switching the SD card, or by transferring it over FTP using ftbrony) uncart is open-source and available at GitHub, so you can look at the source code and suggest improvements via pull requests. If you happen to use any uncart code in your projects, make sure to follow the terms of the GPLv2 appropriately. Please take this seriously instead of giving us reasons to keep future work closed! Usage: - Run Brahma to get control over the ARM9 core - Load uncart into it and follow the instructions Credits go to: - archshift: Code cleanups, and general polishing of my PoC dumper - Lioncash: Code cleanups - me: Getting the project started, on-the-fly decryption - Normmatt: providing the core dumping code and doing tons of reverse-engineering work - yuriks: compatibility enhancements NOTE: auto-decryption is still pending review, but should be merged into the main builds soon (tm). For the time being, you can build PR #7 manually. |
| neobrain |
| ||
|
Member Normal user Level: 10 Posts: 13/17 EXP: 4112 Next: 302 Since: 11-15-14 Last post: 3097 days ago Last view: 2586 days ago |
This is looking nice, thanks for publishing it! |
| neobrain |
| ||
|
Member Normal user Level: 10 Posts: 14/17 EXP: 4112 Next: 302 Since: 11-15-14 Last post: 3097 days ago Last view: 2586 days ago |
New nightly builds (August 19, 2015)
Windows x64 binaries: revision c3d7977888cd Windows x86 binaries: revision c3d7977888cd |
| neobrain |
| ||
|
Member Normal user Level: 10 Posts: 15/17 EXP: 4112 Next: 302 Since: 11-15-14 Last post: 3097 days ago Last view: 2586 days ago |
The current situation is that future releases of devkitARM will ship with picasso by default, whereas nihstro will continue to be actively developed outside of that. For instance, I just recently added an ".include" feature similar to C's "#include" directive.
As you point out, nihstro indeed is intended to be more homebrew developer friendly than other shader assemblers, and I still think it succeeds in doing so. I'm also very much open for suggestions and/or "first time use" experience reports. As for examples: ctrulib ships with a number of examples using picasso, whereas nihstro includes its own bunch of examples (see the examples/assember directory). Ultimately you can get any of the examples working with either assembler - it's just that each project hardcodes its respective "default assembler" in the Makefile, which you could easily change if need be, however. As for removing the boost dependency, that's unfortunately not going to happen because it helps me a lot in ultimately creating a better program. Also, replacing the boost portions would waste a lot of time that I could spend on adding new features instead. |
| neobrain |
| ||
|
Member Normal user Level: 10 Posts: 16/17 EXP: 4112 Next: 302 Since: 11-15-14 Last post: 3097 days ago Last view: 2586 days ago |
https://twitter.com/smealum/status/656947650999156736  |
| neobrain |
| ||
|
Member Normal user Level: 10 Posts: 17/17 EXP: 4112 Next: 302 Since: 11-15-14 Last post: 3097 days ago Last view: 2586 days ago |
Hi,
In the past few days, I've been working on a new tool called braindump. I'd hereby like to announce a pre-release of the tool, meant to help people interested in ROM hacking to get set up faster. WHAT'S BRAINDUMP? braindump is a 3DS userland homebrew application that can dump decrypted contents (both ExeFS and RomFS) of any 3DS game. This includes cartridge games, but also content obtained from the eShop and even some system applications. Even better yet, braindump runs on any system version since 9.0, including the most recent one. WHAT IS IT USEFUL FOR? Currently, it's for the most only useful for dumping game assets: Graphics, sounds, 3D models, etc. In particular, this should be useful for anyone interested in game modding. braindump's feature set has been covered by uncart and Decrypt9 before, however braindump provides a few major advantages:
DOWNLOAD AND USAGE Binaries are available on GitHub. Note that I'm releasing the tool earlier than planned to make the HANS application shipping with *hax 2.5 more accessible to the broader masses (see below). For now, I'll keep the source code private, but rest assured it will be published as soon as I get to implementing the full feature set that was planned. Extract the archive into your SD card's 3ds folder. The full path to the 3dsx should be "sd:/3ds/braindump/braindump.3dsx", and next to that file there should be an xml file. braindump must be launched from 3ds_hb_menu (3dslink or CIAs won't work). Upon launching, you will be asked for a target title - this is the application that will be dumped. Note that the tool will start dumping immediately after selecting the target title, so better make sure you have enough free space on your SD card. Sorry for the lack of further instructions -- I'd rather get this thing out ASAP to make it useful for the more knowledgeable people. I'm sure we can clear up any confusion in the next few days, though
FREQUENTLY ASKED QUESTIONS How to ROM hack?! At this stage, I cannot give you full instructions on how to mod a game, but here are some quick hints:
Can I use the dumps with Citra? Unfortunately, not yet. I have a question and it isn't answered here!!!11 Give me a break, I wrote the FAQ 5 minutes before publishing this post!
FUTURE PLANS There's a lot more to come in braindump than just dumping ExeFS and RomFS. Stay tuned, I guess
|
| Main - Posts by neobrain |
|
Page rendered in 0.039 seconds. (2048KB of memory used) MySQL - queries: 22, rows: 93/93, time: 0.006 seconds. ![[powered by Acmlm]](img/poweredbyacmlm.png "powering nostalgia")
Acmlmboard 2.064 (2018-07-20)© 2005-2008 Acmlm, Xkeeper, blackhole89 et al. |