There are multiple mathematical formulas and algorithms involved in both the calculation of a public key from a private key, and from a public key to a bech32 address.
I would estimate that it would take you a few decades to do either of these algorithms. With a computer to do some precomputation, you can probably do it in a few years.
For public key from a private key, Andrew Poelstra has a pretty good post describing it here:
https://bitcointalksearch.org/topic/m.15594185 along with a precomputed table for you.
For the Bech32 calculation, you will need to be able to do two different hash functions: SHA256 and RIPEMD160. Then you also need to calculate the checksum which uses a BCH error correcting code.
For SHA256, Ken Shirriff has a pretty good
blog post about how to do SHA256 by hand for mining. It's the same operations, just on different data. Also, you are doing only 1 SHA256 computation, not 2 as mining requires. You perform the SHA256 on your serialized public key in compressed form.
For RIPEMD160, I don't think anyone has really explained how to do by hand (and I don't really want to be cause it is long and takes a lot of time which I don't have). The algorithm is described in
this paper with pseudocode given in Appendix A. It is similar to SHA256 in that the message is broken up into chunks which are XOR'd initially with some initial values and then later with the previous chunk.. You would like use a similar method as described in the SHA256 blog post but with the modifications to be able to do RIPEMD160. You perform RIPEMD160 on the above SHA256.
For calculating the checksum, you use the algorithm described under the Checksum section
in the BIP. The gist of it is that, given a list of numbers, you apply multiple polynomials on all of the numbers and the "sum" of the results is the checksum (it's a lot more complicated than that and I don't remember all of the details). The python code given in the bech32_polymod() function describes how to do this. Note that
^= in python means XOR, not exponentiation.
To calculate the final bech32 string, you first need to convert the hash160 into a list of numbers usable for bech32's checksum calculation. You do this by splitting up the 160 bit hash into 5 bit chunks. Each 5 bit chunk is then a number so you now have a list of 5 bit numbers. You prepend to that the witness version, which is 0, so the list now starts with the number 0. Then you prepend to that an expansion of the human readable part of a bech32 string,
bc. This is expanded by using the bech32_hrp_expand() function in the python code given in the BIP. You will get a list of numbers like so:
[3, 3, 0, 2, 3]. So the resulting list of numbers will be 39 numbers in length and begin with
[3, 3, 0, 2, 3, 0, .... Lastly you append six 0's to the end of the list which represent the positions that the checksum will go in. Now you do the bech32_polymod() function on this list of numbers and the resulting value you get back is the checksum.
Now you must convert all of these numbers to characters. You do this by breaking the checksum into 5 bit chunks to get a list of 6 numbers. Then, given the list of numbers containing the witness version number, the hash160, and the checksum, convert each number to the corresponding character using the lookup table described in the BIP. Note that you do not need to do this to the expanded human readable part. The resulting bech32 address is the human readable part concatenated to the character '1' concatenated to the characters for the witness version number, the hash160, and the checksum. So it will begin with
bc1....
Regarding the human readable part expansion, that is done because the human readable part can be any ascii character and the numbers in the list of numbers need to be in the interval [0, 32). However ASCII can have numbers outside of that interval, so the expansion is done to have the higher bits, then the lower bits so it all fits in the interval.
Bech32 would probably only really take a few days / weeks. You just need to be careful to not make a mistake. The thing that takes a while would be privkey to pubkey.