4dsdev
Views: 614,049 Main | Rules/FAQ | Memberlist | Active users | Last posts | Calendar | Stats | Online users | Search 11-23-17 03:36 PM
Guest:

Main - Posts by neobrain


neobrain
Posted on 11-15-14 06:28 AM, in 3DS Shader Tools - nihstro (rev. 8 of 08-19-15 07:19 AM) Link | #22
WHAT IS NIHSTRO?
nihstro is a collection of tools for dealing with 3DS shader developement and reverse engineering. Currently it accommodates an assembler and a disassembler. The assembly shading language used by nihstro follows a public specification.

Included with nihstro is a description of the shader assembly language and a concrete list instruction set reference. Additionally, two very simple example applications are included to illustrate usage (shaders are bundled in the data subdirectories).

nihstro is written in C++11 and requires the Boost libraries. You can read about the details on the GitHub page.

BINARIES
Windows binaries can be downloaded here (update: new version below!). The Readme contains basic usage information.

REPORTING ASSEMBLER BUGS
Given its early development state, there may be a number of quirks with the assembler, in particular:
* Code which you would intuitively expect to work might not compile, or compile in an unexpected way
* Statements which are valid according to the specification or the instruction set reference may be rejected as invalid, or generate an incorrect shader binary.

If you encounter any of these issues or have any other suggestions to make, please report them in this thread (and include an example shader source and the output shbin in that report). You can also create an issue on GitHub.

neobrain
Posted on 11-15-14 06:40 AM, in Vertex shader ops investigation Link | #23
I mentioned this on IRC already, but for reference: I think talking about this in a shader language just adds a second level of confusion, hence I suggest providing the underlying bytecode in hex along to each assembly line. That's probably a bit more effort, but at least we then can be sure that if something "odd" is happening it is because of the GPU, rather than due to some misdesign of the shader assembler.

For example, I think one issue with what you explained out is that you try to assign a floating point register to the second source argument of the mul instruction. However, as far as I am aware src2 only has 5 bits available for addressing registers, which is why 0x25 cannot be indexed by that (contrary to src1, which has 7 bits). Obviously, this issue only manifests itself in aemstro's shading language, because had you worked with bytecode directly, you wouldn't even have thought of trying to assign 0x25 to src2.

By the way, nihstro is aware of this limitation and hence swaps the values of src1 and src2 if necessary.

neobrain
Posted on 11-15-14 06:51 AM, in CHIP-3DS: CHIP-8 emulator for the 3DS [ARM11] Link | #24
Just gave this a quick try - seems to work fine on my 3DS :)

neobrain
Posted on 11-15-14 09:47 AM, in 3DS Shader Tools - nihstro Link | #26
Fwiw, I just uploaded a Windows binary (link in the first post).

neobrain
Posted on 12-21-14 06:05 PM, in PICA fragment lighting Link | #70
Nice findings so far :)

If you haven't already, you should definitely read the publications "A Real-Time Configurable Shader Based on Lookup Tables" (Ohbuchi & Unno) and "Shading by Quaternion Interpolation" (Hast). Those two papers basically give away the whole algorithm used in hardware, and will give you a hint of what to look for.

neobrain
Posted on 02-18-15 01:42 PM, in So what are we providing with devkitARM for shaders? (rev. 2 of 02-18-15 01:49 PM) Link | #87
Just a quick overview:

smea's aemstro:
- scope: assembler + disassembler
- dependencies: python3 (runtime)
- history: was the first shader (dis)assembler to be published and kind of a playground for reverse-engineering shaders
- usability: it works, but isn't really tailored for easy or convenient use.
- prospects: I think given the alternatives and smea's amount of free time, there are no plans on making it more usable for homebrew developers.

neobrain's nihstro:
- scope: assembler + disassembler + C++-library
- dependencies: boost (compile-time), C++11 (compile-time), CMake (compile-time)
- history: created because a shader disassembler had to be written for Citra anyway; the assembler component is the result of me wanting to try out boost::spirit.
- usability: shader language is design for convenient usage
- prospects: actively maintained, but the assembler hasn't received any work recently, mostly because of the seemingly general lack of interest in GPU homebrew development.

fincs's picasso:
- scope: assembler
- dependencies: autotools, TDM-GCC (on Windows; compile-time)
- history: written after aemstro and nihstro because both were considered to be lacking in different aspects by the author
- usability: follows syntax similar to nihstro
- prospects: it hasn't been updated in a while. It's only an assembler.

My (biased) opinion is that aemstro isn't going anywhere and picasso is built on arguably inferior technology, but probably "good enough" for most people. As for nihstro, I would probably put the assembler portion higher on my TODO list if more people actually started creating 3DS homebrew using the GPU.

In any case, picasso is probably the only thing suitable for inclusion into devkitARM, because including nihstro (apparently) means including boost, which is (understandably) unacceptable for the DKA maintainers. None of that is relevant for homebrew developers though, since those will probably use a binary distribution anyway. However, I don't think we desperately need to have a shader assembler included in DKA if that makes for a less pleasant development experience, especially since we don't really know yet which of the three projects will remain active in the long run.

If people insist on shipping *something* with DKA, I would say let things cool down for another few months and see which of the three projects is most advanced by then and/or most commonly used by homebrew developers. That's better than imposing some default onto people (who will probably think it's the only option available) and making them convert over half their code base when they realize the external tools end up being superior. If nihstro turns out to be the favorite, then I guess it would be best to simply not ever include a shader assembler in DKA.

neobrain
Posted on 03-06-15 09:57 AM, in 3DS Shader Tools - nihstro (rev. 2 of 03-06-15 09:58 AM) Link | #101
Just put out an official 0.1 pre-release, link in the first post!

Quoting announcement from gbatemp:
Hi there,

Ever since the release of ninjhax, only few homebrew applications have been developed, which to a large degree probably stems from the lack of tools for convenient GPU programming. This is a pity, especially since the PICA200 could achieve quite nice things given its unique feature set compared to other GPUs targeted at the mobile market: There are extensions for fragment lighting, procedural textures, gaseous object rendering, and other things. While none of these are understood by the homebrew community (yet!), you can see that there is a lot of potential.

Over the past months I've hence been working on a shader assembler (as part of my tool suite nihstro), which has now become advanced enough to be usable for the general 3DS homebrew development. By putting out a prerelease, I'm hoping to get some feedback on the used shader assembly syntax and ease of use.

The shader assembler currently supports most of the instructions listed on 3dbrew, excluding MOVA, MAD, EMIT and SETEMIT. I'm of course planning to add support for these in the future.

If you're interested, head over to the nihstro thread on our homebrew community board 4dsdev for more information and Windows binaries.

Other than that, any feedback on nihstro is welcome!

Regards,
neobrain

neobrain
Posted on 03-08-15 07:52 AM, in 3DS Shader Tools - nihstro Link | #111
Posted by Tcm0
Could you please add some samples like how to display backgrounds (if that's possible) and how to texture triangles?


Unfortunately I don't have too much time on my hands to come up with new examples, so any contributions would be welcome there. As for texturing triangles, the cube demo will show you how to do that already. Displaying backgrounds might be as simple as rendering a textured quad (= two triangles) spanning the whole screen; something more sophisticated is known as "sky boxes", and should be easy enough to find using Google.

neobrain
Posted on 03-13-15 11:31 AM, in Board updated Link | #114
And finally, the annoying "new" icon on the "Unbricking and fixing" section is gone :D

neobrain
Posted on 05-17-15 09:47 AM, in PICA fragment lighting (rev. 3 of 05-17-15 09:57 AM) Link | #128
Normmatt found some more info and uploaded it to https://gist.github.com/Normmatt/9893b46b1bca86af3fd4#file-pica_fragment_light . In particular, this section looks interesting:
Posted by that link
0x002b8977: DW_TAG_enumerator [26]
0x002b8978: DW_AT_name [DW_FORM_string] ("PICA_DATA_SAMPLER_D0")
0x002b898d: DW_AT_const_value [DW_FORM_indirect] (0x00)

0x002b898f: DW_TAG_enumerator [26]
0x002b8990: DW_AT_name [DW_FORM_string] ("PICA_DATA_SAMPLER_D1")
0x002b89a5: DW_AT_const_value [DW_FORM_indirect] (0x01)

0x002b89a7: DW_TAG_enumerator [26]
0x002b89a8: DW_AT_name [DW_FORM_string] ("PICA_DATA_SAMPLER_FR")
0x002b89bd: DW_AT_const_value [DW_FORM_indirect] (0x03)

0x002b89bf: DW_TAG_enumerator [26]
0x002b89c0: DW_AT_name [DW_FORM_string] ("PICA_DATA_SAMPLER_RB")
0x002b89d5: DW_AT_const_value [DW_FORM_indirect] (0x04)

0x002b89d7: DW_TAG_enumerator [26]
0x002b89d8: DW_AT_name [DW_FORM_string] ("PICA_DATA_SAMPLER_RG")
0x002b89ed: DW_AT_const_value [DW_FORM_indirect] (0x05)

0x002b89ef: DW_TAG_enumerator [26]
0x002b89f0: DW_AT_name [DW_FORM_string] ("PICA_DATA_SAMPLER_RR")
0x002b8a05: DW_AT_const_value [DW_FORM_indirect] (0x06)

0x002b8a07: DW_TAG_enumerator [26]
0x002b8a08: DW_AT_name [DW_FORM_string] ("PICA_DATA_SAMPLER_SP")
0x002b8a1d: DW_AT_const_value [DW_FORM_indirect] (0x08)

0x002b8a1f: DW_TAG_enumerator [26]
0x002b8a20: DW_AT_name [DW_FORM_string] ("PICA_DATA_FRAG_LIGHT0_SAMPLER_SP")
0x002b8a41: DW_AT_const_value [DW_FORM_indirect] (0x08)

0x002b8a43: DW_TAG_enumerator [26]
0x002b8a44: DW_AT_name [DW_FORM_string] ("PICA_DATA_FRAG_LIGHT1_SAMPLER_SP")
0x002b8a65: DW_AT_const_value [DW_FORM_indirect] (0x09)

0x002b8a67: DW_TAG_enumerator [26]
0x002b8a68: DW_AT_name [DW_FORM_string] ("PICA_DATA_FRAG_LIGHT2_SAMPLER_SP")
0x002b8a89: DW_AT_const_value [DW_FORM_indirect] (0x0a)

0x002b8a8b: DW_TAG_enumerator [26]
0x002b8a8c: DW_AT_name [DW_FORM_string] ("PICA_DATA_FRAG_LIGHT3_SAMPLER_SP")
0x002b8aad: DW_AT_const_value [DW_FORM_indirect] (0x0b)

0x002b8aaf: DW_TAG_enumerator [26]
0x002b8ab0: DW_AT_name [DW_FORM_string] ("PICA_DATA_FRAG_LIGHT4_SAMPLER_SP")
0x002b8ad1: DW_AT_const_value [DW_FORM_indirect] (0x0c)

0x002b8ad3: DW_TAG_enumerator [26]
0x002b8ad4: DW_AT_name [DW_FORM_string] ("PICA_DATA_FRAG_LIGHT5_SAMPLER_SP")
0x002b8af5: DW_AT_const_value [DW_FORM_indirect] (0x0d)

0x002b8af7: DW_TAG_enumerator [26]
0x002b8af8: DW_AT_name [DW_FORM_string] ("PICA_DATA_FRAG_LIGHT6_SAMPLER_SP")
0x002b8b19: DW_AT_const_value [DW_FORM_indirect] (0x0e)

0x002b8b1b: DW_TAG_enumerator [26]
0x002b8b1c: DW_AT_name [DW_FORM_string] ("PICA_DATA_FRAG_LIGHT7_SAMPLER_SP")
0x002b8b3d: DW_AT_const_value [DW_FORM_indirect] (0x0f)


This seems to confirm your findings in your post about LUTs. According to DMP's research papers, SP models a spotlight if the proper lookup function (namely -L*P) is used, so it should probably be renamed to "spotlight" indeed.

Some further information on the LUTs: They take a one-dimensional index, which is user-configurable. The index is calculated from the set (N*H, V*H, L*N, -L*P, cos Phi), with P the spotlight direction, Phi "the angle between the projection of H onto the tangent plane and the tangent vector T". I suppose the LUT index is set up by PICA_REG_FRAG_LIGHT_LUTINPUT, i.e. GPU register 0x1D1 (this is pure speculation based on that register's name, though).

Some of the vectors in these formulas are likely configured via GPU registers, while others are vertex shader outputs. In particular, the view vector V is given by the "view" output vertex attribute semantic, while the normal vector N and tangent vector T are calculated from the "quaternion" output vertex attribute semantic. H should simply be something like (V+N)/2 (or its normalization).

The precise calculation of N and T happens by using the quaternion to transform the surface-local space into eyespace (which is really just a simple rotation, and hence can indeed be encoded in a unit quaternion). In surface-local space, N and T always take fixed values (e.g. N=(0,0,1) and T=(0,1,0)). Hence all required information about the eye-space values really is encoded in the quaternion. Side note: Quaternion-vector multiplication is a linear operation, hence to get per-fragment values of N and T it should be sufficient to linearly interpolate the quaternion.

neobrain
Posted on 06-02-15 04:06 PM, in 3DS Shader Tools - nihstro (rev. 4 of 08-19-15 07:19 AM) Link | #165
New Windows x64 binaries: revision c3d7977888cd
New Windows x86 binaries: revision c3d7977888cd


New builds below.

neobrain
Posted on 06-07-15 11:16 AM, in uncart -- dumping decrypted 3DS game cartridges (rev. 2 of 06-07-15 11:18 AM) Link | #181
Ever wanted to backup your 3DS games for usage with Citra, but didn't feel like resorting to piracy-oriented solutions like Gateway? Looking for a less painful way to decrypt game dumps, which doesn't require a gazillion of different tools? Wait no longer, here's uncart!

uncart is a tool for dumping 3DS game cartridges that I have been working on with a number of other people over the past few weeks. It will dump any 3DS game cartridge to the SD card, from which you can transfer it to your computer (by just switching the SD card, or by transferring it over FTP using ftbrony)

uncart is open-source and available at GitHub, so you can look at the source code and suggest improvements via pull requests. If you happen to use any uncart code in your projects, make sure to follow the terms of the GPLv2 appropriately. Please take this seriously instead of giving us reasons to keep future work closed!

Usage:
- Run Brahma to get control over the ARM9 core
- Load uncart into it and follow the instructions

Credits go to:
- archshift: Code cleanups, and general polishing of my PoC dumper
- Lioncash: Code cleanups
- me: Getting the project started, on-the-fly decryption
- Normmatt: providing the core dumping code and doing tons of reverse-engineering work
- yuriks: compatibility enhancements

NOTE: auto-decryption is still pending review, but should be merged into the main builds soon (tm). For the time being, you can build PR #7 manually.

neobrain
Posted on 07-01-15 07:20 AM, in CakeHax - Open Source ARM9 loader Link | #241
This is looking nice, thanks for publishing it! :)

neobrain
Posted on 08-19-15 07:18 AM, in 3DS Shader Tools - nihstro (rev. 2 of 08-19-15 07:18 AM) Link | #342
New nightly builds (August 19, 2015)

Windows x64 binaries: revision c3d7977888cd
Windows x86 binaries: revision c3d7977888cd

neobrain
Posted on 08-23-15 07:36 AM, in So what are we providing with devkitARM for shaders? (rev. 4 of 08-23-15 07:38 AM) Link | #366
The current situation is that future releases of devkitARM will ship with picasso by default, whereas nihstro will continue to be actively developed outside of that. For instance, I just recently added an ".include" feature similar to C's "#include" directive.

As you point out, nihstro indeed is intended to be more homebrew developer friendly than other shader assemblers, and I still think it succeeds in doing so. I'm also very much open for suggestions and/or "first time use" experience reports.

As for examples: ctrulib ships with a number of examples using picasso, whereas nihstro includes its own bunch of examples (see the examples/assember directory). Ultimately you can get any of the examples working with either assembler - it's just that each project hardcodes its respective "default assembler" in the Makefile, which you could easily change if need be, however.

As for removing the boost dependency, that's unfortunately not going to happen because it helps me a lot in ultimately creating a better program. Also, replacing the boost portions would waste a lot of time that I could spend on adding new features instead.

neobrain
Posted on 10-22-15 07:58 PM, in Eshop Link | #545
https://twitter.com/smealum/status/656947650999156736 ;)

neobrain
Posted on 10-26-15 07:14 PM, in braindump prerelease - dump game contents on any system version! Link | #578
Hi,

In the past few days, I've been working on a new tool called braindump. I'd hereby like to announce a pre-release of the tool, meant to help people interested in ROM hacking to get set up faster.

WHAT'S BRAINDUMP?
braindump is a 3DS userland homebrew application that can dump decrypted contents (both ExeFS and RomFS) of any 3DS game. This includes cartridge games, but also content obtained from the eShop and even some system applications. Even better yet, braindump runs on any system version since 9.0, including the most recent one.

WHAT IS IT USEFUL FOR?
Currently, it's for the most only useful for dumping game assets: Graphics, sounds, 3D models, etc. In particular, this should be useful for anyone interested in game modding.

braindump's feature set has been covered by uncart and Decrypt9 before, however braindump provides a few major advantages:
  • it runs on any system version newer than 9.0, including the most recent one. uncard and Decrypt9 are limited to system versions 9.2 or below.
  • it dumps cartridge games, eShop content, and (some) system applications in a uniform way. All three of these had to be decrypted with different commands before.
  • it's more convenient, since it can be launched directly from the Homebrew Launcher and only requires one operation step from starting to the decrypted content. uncart and Decrypt9 had to be launched through a separate program, and after dumping a game you had to restart your console and run another chain of commands to actually decrypt things.

DOWNLOAD AND USAGE
Binaries are available on GitHub. Note that I'm releasing the tool earlier than planned to make the HANS application shipping with *hax 2.5 more accessible to the broader masses (see below). For now, I'll keep the source code private, but rest assured it will be published as soon as I get to implementing the full feature set that was planned.

Extract the archive into your SD card's 3ds folder. The full path to the 3dsx should be "sd:/3ds/braindump/braindump.3dsx", and next to that file there should be an xml file. braindump must be launched from 3ds_hb_menu (3dslink or CIAs won't work). Upon launching, you will be asked for a target title - this is the application that will be dumped. Note that the tool will start dumping immediately after selecting the target title, so better make sure you have enough free space on your SD card.

Sorry for the lack of further instructions -- I'd rather get this thing out ASAP to make it useful for the more knowledgeable people. I'm sure we can clear up any confusion in the next few days, though ;)

FREQUENTLY ASKED QUESTIONS
How to ROM hack?!
At this stage, I cannot give you full instructions on how to mod a game, but here are some quick hints:
  • Dump the game contents using braindump on your 3DS. This will place the files exefs.bin and romfs.bin on your SD card (in a subdirectory with a 16-digit number as its name)
  • On your PC, extract the two files using ctrtool. Use "ctrtool --intype=romfs --romfsdir=romfsdir romfs.bin" and "ctrtool --intype=exefs --exefsdir=exefsdir --decompresscode exefs.bin" to unpack the romfs and exefs, respectively
  • Game modders will be interested in the contents extracted to romfsdir. Modify whatever you like, and repack the contents using 3dstool
  • Put the new romfs binary on your SD card. Start HANS on your 3DS and point it to the modded game, and make it replace the romfs with your new image. Note that for HANS to detect the romfs binary, you'll need to remove the first 0x1000 bytes from the romfs binary.

Can I use the dumps with Citra?
Unfortunately, not yet.

I have a question and it isn't answered here!!!11
Give me a break, I wrote the FAQ 5 minutes before publishing this post! ;)

FUTURE PLANS
There's a lot more to come in braindump than just dumping ExeFS and RomFS. Stay tuned, I guess ;)


Main - Posts by neobrain

Page rendered in 0.051 seconds. (2048KB of memory used)
MySQL - queries: 22, rows: 93/93, time: 0.030 seconds.
[powered by Acmlm] Acmlmboard 2.064 (2017-11-20)
© 2005-2008 Acmlm, Xkeeper, blackhole89 et al.