Shainer's SiteRandom stuff I find interesting or I am working on. At some point I will move all the content from my old website, giudoku.sourceforge.net, here.
http://shainer.github.io/
Mon, 11 Sep 2017 23:30:45 +0000Mon, 11 Sep 2017 23:30:45 +0000Jekyll v3.5.2Finding solutions to weird equations<p>A while back, on social media, I found a link to <a href="https://www.quora.com/How-do-you-find-the-positive-integer-solutions-to-frac-x-y%2Bz-%2B-frac-y-z%2Bx-%2B-frac-z-x%2By-4/answer/Alon-Amit?share=1">an interesting post on solving a particular mathematical equation</a>. Now I know this description might not sound very enticing; people solve equations every day! But, some equations carry more meaning than what appears at first glance.</p>
<p>The equation has a pretty symmetric structure and a simple definition:</p>
<p><img src="http://shainer.github.io/images/equation.png" alt="The equation" /></p>
<p>And we want to find the <strong>positive</strong> solution or solutions to this equation. How? As will be clear, brute forcing won’t
be enough, so we have to study the properties of this equation and come up with ingenious methods.</p>
<p>The post then explains how we are dealing with a 3-degree equation that has at least one rational (but not positive) solution.
This means the equation describes an <strong>elliptic curve</strong>: by mean of complex (and quite boring) transformations, we can
express it in the usual elliptic curve form, which is</p>
<div class="highlighter-rouge"><pre class="highlight"><code>y^2 = x^3 + ax + b
</code></pre>
</div>
<p>with a and b rational coefficients of the curve. At this point the post author provides one solution for the equation,
the point (-100, 260). By simple math, the corresponding values of the initial unknowns are computed as 4, -1 and 11. So this
is not good, because it’s not a positive solution. Now, we can make use of some properties of elliptic curves to find new
solutions to test for positivity.</p>
<p>In particular, elliptic curves are closed under addition: adding two points P and Q on the curve yields another point on
the curve. Such point is found by drawing a line connecting the two points, looking for a third point where the line intersects
the curve, and taking the point that is symmetric to this on the x axis (which will also belong to the curve).</p>
<p>Now, if we add our initial point (-100, 260) to itself, finding 2P, 3P, etc… and every time we test the new point to see if
the unknowns are positive. We have to arrive to 9P to finally find the following solution to the equation:</p>
<p>a=154476802108746166441951315019919837485664325669565431700026634898253202035277999,
b=36875131794129999827197811565225474825492979968971970996283137471637224634055579,
c=4373612677928697257861252602371390152816537558161613618621437993378423467772036</p>
<p>well those are huge numbers, so you can see why employing brute force wouldn’t have helped here.</p>
<p>But why did we pick a point and started adding it to itself? Well it turns out this point is the one and only <em>generator</em>
of the curve, so by adding it to itself enough time you are able to find any other non-infinity point on the curve. So if the positive solution
existed, we were bound to find it with this method.</p>
<p>The final interesting thing is that the number of digits in the first positive solution we can find with the generation
method depends on the coefficient on the right side of the equation (here, 4). The bigger this coefficient, the bigger the number
of digits is going to be. How bigger? Can we find a correlation between the two quantities?</p>
<p>Unfortunately, we cannot. This is because there is no algorithm for finding an integer solution to a Diophantine equation, so
naturally we cannot make statements about the property of such solution. We cannot even know if one will exist in advance.
Thanks to this post, I am now aware of the <a href="https://en.wikipedia.org/wiki/Hilbert%27s_tenth_problem">story behind this</a>, which
I might explore in more details in a future post.</p>
<h2 id="extra-notes">Extra notes</h2>
<p>I am temporarily without my main laptop, using a Chromebook (long story). So I don’t have access to my usual writing environment and I wrote this post entirely on Github. I have to say that the general flow is quite good: it’s easy to upload a bunch of new files to some location and create a commit, and the Markdown editor is decent with live preview. I still prefer Atom for that though :)</p>
<p>Also, embedding LateX in Markdown, at least for Github Pages, is still a messy affair. Perhaps one day I’ll find the patience to figure out which method actually works and does not require complex HTML code. That day is not today, sorry, you get the equation as a PNG image cut from a screenshot :D</p>
Tue, 12 Sep 2017 00:30:00 +0000
http://shainer.github.io/math/2017/09/12/finding-solutions-to-weird-equations.html
http://shainer.github.io/math/2017/09/12/finding-solutions-to-weird-equations.htmlmathForging RSA signatures<p>As I vaguely promised some weeks ago, going back to solve the crypto challenges I had missed in the sets before the eigth one, I found yet another interesting problem to talk about: exploiting several weaknesses to <a href="http://cryptopals.com/sets/6/challenges/42">forge a RSA signature</a>.</p>
<h2 id="preconditions">Preconditions</h2>
<p>The following conditions are necessary for this attack to work:</p>
<ol>
<li>RSA is used to generate and verify digital signatures;</li>
<li>The keypair generation is lazy, and sets the public exponent e to 3;</li>
<li>The signature verification is also a bit lazy; I’ll describe how later.</li>
</ol>
<p>#1 is nothing weird: digital signatures need some form of asymmetric encryption and RSA is the most popular choice.</p>
<p>#2 can happen in practice due to how keypair generation works: the two internal parameters for RSA, q and p, need to be primes, large enough to make the factorization of N = pq hard, and such that (p - 1) and (q - 1) are either coprimes or don’t have a lot of common factors after 2. A common way to achieve this is to set e, the “exponent” part of the final public key, to some small prime like 3, and then derive p, q and d (the exponent of the private key) from that. Having a small value for e also makes encryption and decryption quite easy, since we work with “small” numbers.</p>
<p>The full standard to compute signatures with RSA is described in <a href="https://tools.ietf.org/html/rfc2313">RFC 2313</a>. In short, the message is first hashed (most common algorithms are supported), then the following bytes block is generated:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>00 01 FF .. FF 00 ASN.1 HASH
</code></pre>
</div>
<p>where ASN.1 is a byte string identifying the hash algorithm used (see <a href="https://www.ietf.org/rfc/rfc3447.txt">RFC 3447</a>), and there’s as many FF bytes as needed to make the total size equal to the size in
bytes of N, the modulus of the RSA keys. Note that N is part of both the public and the private RSA key. The block is then converted to the corresponding integer and encrypted with the private key; optionally, the result is converted again to an hex string or similar representation.</p>
<p>Now we can see where mistake #3 can come from: if I verify a RSA signature using, for example, regular expressions, it’s easy to check that there’s one or more FF bytes in the padding zone, but not checking that there’s exactly the number I expect. Furthermore, I might not check that there’s nothing else in the signature after the hash. Note that this bites me even if I separately check that the
total signature length is as expected.</p>
<p>If all these conditions are there, the attacker is able, without any knowledge of the private key, to forge a RSA signature for pretty much any message, and have it accepted by the verifier.</p>
<h2 id="how-the-forgery-works">How the forgery works</h2>
<p>If I am verifying a signature, I am decrypting with the public key, which means this operation is performed:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>D = EncryptedSignature ** e mod N
</code></pre>
</div>
<p>if e is equal to 3, it’s quite possible that <code class="highlighter-rouge">EncryptedSignature ** 3</code> ends up being smaller than N, therefore the modulo operation does not change the result. So, if we forge a block that satisfies only the conditions we know the system checks for, and also corresponds to a perfect cube, we can pass the cube root as a signature to such a verification system, and it will be accepted as valid. How does that happen?</p>
<p>Let’s take a block with this format:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>00 01 FF 00 ASN.1 HASH GARBAGE
</code></pre>
</div>
<p>Now let’s take the sub-block composed of the last 00 byte, the ASN.1 code, and the hash. If SHA-256 is used for the hash, the total size is 52 bytes. We then convert this block into an integer, which we call D.</p>
<div class="highlighter-rouge"><pre class="highlight"><code>Block = '00 ASN.1 HASH'
D = int(Block)
N = 2^len(Block) - D
</code></pre>
</div>
<p>Now let’s say that our RSA key has length 2048 bits; with the format above, there are going to be (2048 - 52 - 3) bits left on the right for garbage. Let’s call this number X. The numeric block is going to be:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>2 ^ (2048 - 15) - 2 ^ (X + len(Hash)) + D * 2^X + garbage
2 ^ (2048 - 15) - N * (2^X) + garbage
</code></pre>
</div>
<p><strong>Edit</strong>: the second equation follows from the first quite easily if you remember that according to the definition above, <code class="highlighter-rouge">N = 2^len(Block) - D</code>, and therefore, <code class="highlighter-rouge">D = 2^len(Block) - N</code>.</p>
<p><strong>Disclaimer</strong>: this is where I get lost, sadly. I don’t understand what the 15 bits subtracted from the key size represent. If somebody reads this and wants to send me an email for a more complete explanation, they are welcome to do it!</p>
<p>From empirical evaluation, it seems that for the garbage number you should prefer higher values: the lowest cube roots might end up being encoded in something that does not quite contain the full hash at the end, possibly because we run out of bits to convert before that. This is why my code takes a shortcut and just sets it to the highest number possible given the allowed number of bits.</p>
<p>The final step is computing the cube root and converting the result to an integer (in my case, by rounding down). Here I ran into a limitation of Python: the suggested way to compute a cube root is to elevate the number to the power of (1.0 / 3.0), but this requires the base to be converted to a float, and that does not work for very large integers such as this one. I could have looked at some mathematical library like numpy, but I am reluctant to add too many dependencies; eventually I found a code snippet on the Internet that does the job with the decimal builtin module.</p>
<p>Aaand we are done!</p>
Sun, 20 Aug 2017 17:30:00 +0000
http://shainer.github.io/crypto/2017/08/20/forging-rsa-signatures.html
http://shainer.github.io/crypto/2017/08/20/forging-rsa-signatures.htmlcryptoInteresting C++ features, part 2<p>I have developed in C++ a lot, both at work and outside. So I like to keep updated with the new features and utilities introduced by
new versions or available through common libraries. This post will describe a few new things I have discovered recently.</p>
<p>If you are wondering why this is part 2, I have decided <a href="https://shainer.github.io/c++/opensource/2016/11/13/cpp-errors.html">this post</a> can be
considered as “part 1” of this series. I plan to write more about C++ in the future: I’ll make all the posts numbered and under the
“c++” category.</p>
<h2 id="span">span</h2>
<p>This is not part of the STL, but rather of the Guidelines Support Library, which is any library implementing <a href="https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md">this set of guidelines</a> by the standards committee.</p>
<p>Let’s say that you want to pass an array to a function as a pointer. A basic example:</p>
<div class="language-cpp highlighter-rouge"><pre class="highlight"><code><span class="kt">int</span> <span class="nf">sum</span><span class="p">(</span><span class="kt">int</span><span class="o">*</span> <span class="n">data</span><span class="p">,</span> <span class="kt">int</span> <span class="n">n</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// sum all elements in data.
</span><span class="p">}</span>
</code></pre>
</div>
<p>Now in order for this code to work, we assume n represents the array size, and that it’s
actually correct and does not make us access out-of-bounds memory.</p>
<p>A better way:</p>
<div class="language-cpp highlighter-rouge"><pre class="highlight"><code><span class="kt">int</span> <span class="nf">sum</span><span class="p">(</span><span class="k">const</span> <span class="n">span</span><span class="o"><</span><span class="kt">int</span><span class="o">>&</span> <span class="n">data</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// sum all elements in data.
</span><span class="p">}</span>
</code></pre>
</div>
<p>a <code class="highlighter-rouge">span</code> of the array (which might or might not contain all the elements) is built on-the-fly to
represent a view of the array, and passed to any function. All the information about size are contained
internally.</p>
<p><code class="highlighter-rouge">array_view</code> (which has a more descriptive name) works the same way, but unlike <code class="highlighter-rouge">span</code> it’s a read-only
view of the original array. This is preferrable to ensure no function can write on what is a view of another
data structure, without having to enforce constness instead.</p>
<h2 id="custom-error-codes">Custom error codes</h2>
<p>This was introduced in C++11, but I think it was overshadowed by other more revolutionary features, since you seldom find
trace of it.</p>
<p>There is another blog which describes quite in details <a href="https://akrzemi1.wordpress.com/2017/07/12/your-own-error-code/">how to define your own error code space</a> and <a href="https://akrzemi1.wordpress.com/2017/08/12/your-own-error-condition/">how to write error conditions</a>. It’s no use repeating
the entire content of those posts here, so I’ll make a short summary.</p>
<p>The <code class="highlighter-rouge">std::error_code</code> is a generic interface (used in non-programming sense of the word) to express custom
error codes. Error codes are identified by a <em>number</em> (minus 0, which always means success) and a <em>domain</em> or <em>category</em>; the
latter specifies the types of errors we are dealing with, and is identified by a name.</p>
<p>At a high level, this machinery allows you to construct and use variables of type <code class="highlighter-rouge">std::error_code</code> from the enum values representing the
custom error codes you need. By subclassing <code class="highlighter-rouge">std::error_category</code> you define a custom category,
with some nice functionalities such as associating an error message to each code.</p>
<p>Moving forward, it’s also possible to express complex groupings and conditions on your set of errors by using <code class="highlighter-rouge">std::error_condition</code>.
Let’s say that all of your errors fall into the following sub-categories: <strong>internal errors</strong> (something happens inside your program) and
<strong>external errors</strong> (due to e.g. networking). You can extend the logic of your category with a function that tells you which sub-category
a given code belongs to.</p>
<p>Personal opinion: this is not trivial to use, and requires more boilerplate than usual for the initial set up of codes and categories.
However once that part is done (likely in some utility library) it is incredibly useful: good error handling is
something many applications and libraries miss (and let’s not talk about dealing with <code class="highlighter-rouge">errno</code>…). I am happy to pay the cost to
have sets of well-defined error spaces and codes to deal with.</p>
<h2 id="cppcon">CppCon</h2>
<p>CppCon 2017 is around the corner! I won’t be able to attend, but it’s on my todo list and I am currently spending time watching
talks from previous editions on YouTube. I definitely recommend to keep an eye for this year’s talks.</p>
Sun, 13 Aug 2017 10:00:00 +0000
http://shainer.github.io/c++/2017/08/13/more-cpp-features.html
http://shainer.github.io/c++/2017/08/13/more-cpp-features.htmlc++Cleaning up the Matasano repositories<p>I started the Github repository for the Matasano crypto challenges mostly for myself. It’s not software with actual users, so I didn’t pay a lot of attention to code health or general organization.</p>
<p>But as I kept adding more files, and referencing my code in posts, I realized it was a pretty bad shape and it would benefit from some love. So I went back nand reorganized all the files, making sure viewers (and myself) can easily figure out which Python file is a binary you run to solve a given challenge, and which is a library used to abstract common operations. The README also gives more information about the status of the work and what dependencies the applications have.</p>
<p>Therefore I officially apologize to the people who looked at it or forked it when it was in a much worse shape :-)</p>
<p>This cleanup also had an unintended consequence: I realized I didn’t actually solve all the challenges. I skipped the MD4 one on purpose, as I already explained, but I actually “left for later” another 3-4 challenges, and then completely forgot about this. Well, better late than never, so I am now going back and solving them. Challenge 20 was the first of the list, and I just submitted the solution this morning.</p>
<p>So double yay for code cleanup!</p>
Mon, 07 Aug 2017 12:00:00 +0000
http://shainer.github.io/matasano/cleanup/2017/08/07/cryptopals-code-cleanup.html
http://shainer.github.io/matasano/cleanup/2017/08/07/cryptopals-code-cleanup.htmlmatasanocleanupFish shell<p>Among way more serious topics, there is something trivial I want to share with the world: I officially changed my default shell to <a href="https://fishshell.com">fish</a>.</p>
<p>Why? The autosuggestion features are way more user-friendly. While you are typing a command, you can see a suggestion for the rest of the command, options or files form to the right of your cursor, based on your recent history. Multiple suggestions are easy to browse and select should the first one not be the right one for you. Moreover, completions are derived from the manpages installed on your system, so you get the most useful ones out of the box; by looking atthe repository I believe it is also possible to add more completions if the manpage does not contain everything. Default colouring is nice too.</p>
<p>Fish comes with its own scripting language; I don’t plan to write any Fish script right now, so I haven’t looked at that at all.</p>
<p>For Chakra Linux users, I packaged fish in [desktop], and I plan to keep an eye on new releases to update it regularly.</p>
Sun, 16 Jul 2017 16:20:00 +0000
http://shainer.github.io/linux/2017/07/16/fish-shell.html
http://shainer.github.io/linux/2017/07/16/fish-shell.htmllinuxDiscrete logarithms: a guide<p>I am working through the second challenge in the 8th cryptopals set, and I am already learning something new. Let’s talk about discrete logarithms, what they are and how to compute them.</p>
<p>When preparing a Diffie Hellman key exchange, some parameters must be chosen first:</p>
<ul>
<li>P: a large prime, is the order of a finite <strong>cyclic group</strong>.</li>
<li>G: the <strong>generator</strong> of the cyclic group.</li>
</ul>
<p>G is a generator if for every element of the group, there is a x such that <code class="highlighter-rouge">G^x mod P</code> gives that element of the group.</p>
<p>The shared secret of Diffie Hellman is computed using <strong>modular exponentiation</strong> of the generator. There are algorithms to compute modular exponentiation efficiently, even for numbers that have hundreds of bits. However, the security of Diffie Hellman (among others) depends on the fact that the inverse operation, the <strong>discrete logarithm</strong>, cannot be computed efficiently in the general case, if the parameters are chosen with the right properties.</p>
<p>However, algorithms that are able to do better than the brute force still exist. Let’s talk about two of them: <strong>baby-step, giant-step</strong> and <strong>Pollard’s kangaroo</strong>.</p>
<h2 id="baby-step-giant-step">Baby-step, giant-step</h2>
<p>Recall the formulation of the problem: we want to find x such that</p>
<div class="highlighter-rouge"><pre class="highlight"><code>G^x mod P = B
</code></pre>
</div>
<p>We can rewrite x as</p>
<div class="highlighter-rouge"><pre class="highlight"><code>x = im + j
</code></pre>
</div>
<p>where m is sqrt(P) and i and j are two (unknown) coefficients between 0 and m. So applying some exponential properties the formulation becomes:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>B(G^-m)^i = G^j
</code></pre>
</div>
<p>All operations happen inside the group, so modulo P. Now, we precompute <code class="highlighter-rouge">G^j mod P</code> for all values of j up to m. We store them in a data structure, for easy lookup later; the data structure maps these exponentials with the value of j. A natural choice here is a hash map.</p>
<p>One of the values of j will be the one that makes the above equation true. But we need to find i too. So we brute force it: for all possible i between 0 and m, we compute <code class="highlighter-rouge">B(G^-m)^i mod P</code>. If the result is in the data structure we built initially, we retrieve the corresponding j. We have found both coefficients, and therefore x.</p>
<h3 id="complexity">Complexity</h3>
<p>This is still a brute force algorithm, however, instead of trying all possible x between 0 and P (the trivial brute force), we only try up to m, the square root of P. This reduces the number of operations we perform in the worst case. So in terms of complexity analysis it does not mean much. We also “pay” for this speedup by using more memory to store m key-value pairs.</p>
<h3 id="implementation">Implementation</h3>
<p>I did mention I wanted some practice with Rust, so <a href="https://github.com/shainer/baby-step">here is a Rust implementation of baby-step giant-step algorithm</a>. The code contains efficient implementations of modular inverse (to compute <code class="highlighter-rouge">G^-m</code>), and of modular exponentiation. I have used both several times in the Matasano solutions, but here I took the time to examine, understand and explain them in the comments.</p>
<h2 id="pollards-kangaroo-algorithm">Pollard’s kangaroo algorithm</h2>
<p>This algorithm is used when the discrete logarithm is known to lie in a subrange [a, b] of the group. Of course in the worst case you can set the subrange to [0, P-1], i.e. all elements in the group, but in that case more efficient alternatives exist.</p>
<p>The basic idea is to generate two pseudorandom sequences of elements in the range, and then looking for collisions. The first sequence starts from an element of known discrete logarithm, the second from the element whose logarithm we want to find (B).</p>
<ul>
<li>Define a deterministic function F from elements of the input group to S, a set of integers.</li>
<li>Choose an integer N and compute a sequences of N integers like this:</li>
</ul>
<div class="highlighter-rouge"><pre class="highlight"><code>y0 = G^b mod P
y := y * (G^F(y) mod P) mod P
</code></pre>
</div>
<p>In another variable, usually called the <em>distance</em> D, you store the sum of all the F(y) you computed for the sequence. Also note that for the final element of the sequence, this property holds:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>yFin = G^(b+D) mod P
</code></pre>
</div>
<p>This sequence is the <strong>tame Kangaroo</strong>. It starts from y0, an element whose discrete logarithm is b, the end of our range. From there, we take N jumps to other elements.</p>
<p>Now we define the <strong>wild kangaroo</strong>. The new sequence has the same definition, only we start from B:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>y0 = B
y := y * (G^F(y) mod P)
</code></pre>
</div>
<p>Again we keep track of the “distance” travelled in D’. If the next element of the sequence collides with an element we have seen before, then the discrete logarithm is equal to</p>
<div class="highlighter-rouge"><pre class="highlight"><code>b + D - D'
</code></pre>
</div>
<p>We stop when we have travelled more than <code class="highlighter-rouge">b - a + D</code>. This algorithm does not guarantee that a solution is always found: it is possible to exceed the limit without colliding with the tame kangaroo, even if a discrete logarithm exists.</p>
<p>F controls the size of the jumps you make at each iteration. Bigger jumps give you a better computation time for large numbers, but also increase the probability that you won’t collide. To reduce the risk, N is chosen so that larger outputs of F correspond to a larger N: this causes the first sequence to have more elements.</p>
<h3 id="implementation-1">Implementation</h3>
<p>None of now. Or at least not public: I baked a Python implementation in the solution of the latest challenge I am working on. When I have time I am going to take it out, translate it to Rust, and make it public, but after spending time on the baby-step giant-step algorithm, I don’t feel like it :-)</p>
<h2 id="others">Others</h2>
<p>There are other algorithms that are better than brute force, but none of them run in polynomial time. I have not studied or implemented them so I am not going to talk about them, but <a href="https://en.wikipedia.org/wiki/Discrete_logarithm#Algorithms">Wikipedia has a list</a> if you want to learn more!</p>
Sat, 27 May 2017 17:00:00 +0000
http://shainer.github.io/math/crypto/2017/05/27/discrete-logarithms.html
http://shainer.github.io/math/crypto/2017/05/27/discrete-logarithms.htmlmathcryptoMatasano crypto challenges: set 8<p>Well, the time has come for me to start the 8th set of the crypto challenges.
Actually I have <a href="http://cryptopals.com/sets/7/challenges/55">one remaining
challenge</a> to complete in set 7. I
have started the work and have a good idea of what I am supposed to do, but I
find the actual implementation a bit boring, so I have set it aside for now. The basic
idea, as explained in the challenge itself, is to perform a series of bitwise
operations to tweak bits in a message. After enough tweaks, we should have a new
message that has the same MD4 hash (with a given probability).</p>
<p>Anyway, the authors of the challenges send the 8th set by email, and explicitly
ask not to share the questions publicly. I am not sure if the same applies to
the solutions; I asked them, but have received no reply so far. In the meantime
they will be on a private repository.</p>
<p>I will miss doing the writeups of the most intriguing solutions. So I guess
you’ll see less new posts here. I however occasionally hang out in #cryptopals on
Freenode, and unofficial channel for these challenges.</p>
Sat, 13 May 2017 16:15:00 +0000
http://shainer.github.io/crypto/matasano/2017/05/13/cryptopals-set8.html
http://shainer.github.io/crypto/matasano/2017/05/13/cryptopals-set8.htmlcryptomatasanoAnitya setup completed<p>To recap:</p>
<ul>
<li>I can run an Anitya instance on virtualenv. This is enough for my personal
use. If we decide to run an instance on the Chakra server, I will try running
with docker too.</li>
<li>The RPM library has been made optional; an emulator is available.</li>
<li>I wrote <a href="https://github.com/shainer/the-gitlab-hotness">The GitLab hotness</a>.
This small application waits for new messages on fedmsg, and when the message
concerns a package update, it files a new bug assigned to me on the Chakra
Gitlab instance.</li>
</ul>
<p>I still need to do an end-to-end test to make sure everything is connected
properly, but that should be easy.</p>
<p>The new script also needs to be improved; I want to be able to recognize the
right project to add the bug to, based on which repository the package is hosted
on at the moment. For the moment I am just using the same one regardless.</p>
<p>Thanks to the team for being responsive to my queries and solve the problems I
reported. I don’t think I will talk about this again, given that the bulk of the
work is done.</p>
Sat, 13 May 2017 16:10:00 +0000
http://shainer.github.io/chakra/2017/05/13/anitya-setup.html
http://shainer.github.io/chakra/2017/05/13/anitya-setup.htmlchakraGame theory (or what I am doing these days?)<p>My previous efforts are grinding to a halt a bit. Reason? I have started studying Game Theory and I still haven’t managed to stop!</p>
<p>This isn’t my first encounter with this fascinating science. In university my AI courses briefly touched on the subject,
since it is a useful tool to model interactions between autonomous agents. We did not get very far beyond basic definitions, the Nash equilibrium, and techniques to find it.</p>
<p>So after a recommendation, I picked up <a href="https://www.coursera.org/learn/game-theory-1">the main course on the subject</a> on Coursera. Online courses, as one can expect, vary a lot in quality, even if you pick the subject you love the most. This one,
which has been around for several years now, is definitely on the good end of the spectrum. The lectures do a good job of
explaning the material, provide many real-life scenarios you can use as inspirations, and the associated exercises are fun
to go through even if you do not intend to “pass” the course.</p>
<p>My main criticism is that the lessons are sometimes too short for what they present, so they
remain light on the math concepts and proofs behind the theorems. I have just started <a href="https://www.coursera.org/learn/game-theory-2">the advanced course</a>, by the same university and professors, and for now it seems that it goes way deeper into the math. This second course explains the less “glamorous” aspects of game theory. I had no clue some of them were part of this science to begin with.</p>
<p>The first one is called <a href="https://en.wikipedia.org/wiki/Social_choice_theory">social choice theory</a> and deals with a well-known
problem for anybody who has ever followed an election: given some “candidates” to vote upon, and a set of voters with their preferences over the candidates, how do you pick the winner? There are clearly many ways of “aggregating” the individual preferences into a final choice or ranking, so which one is the most faithful? What properties do you use to define what a good choice looks like? <a href="https://en.wikipedia.org/wiki/Arrow%27s_impossibility_theorem">Arrow’s theorem</a>, the main result in this field, is definitely going to be a surprise!</p>
<p>What’s next? Well I still have to find out myself…</p>
<p>In the meantime, thanks to a very productive conversation with the Anitya team (started by my post), I learned that they are working on removing the dependency on the RPM library, and to improve the login mechanism. I decided it’s better to leave the experts to this: implementing a versioning function that is general enough to apply to as many release systems as possible is definitely a long and complex task, and the team has already started the discussions around it.</p>
<p>While I wait for their code to be complete, I forked the repository and replaced the dependency on RPM with a much simpler version that “works for now” (but is in no way suitable for real use). Now I can run Anitya on my machine, using virtualenv, and add projects to it. The next step is to install fedmsg, let Anitya publish notifications for new releases on the designated topic, and attach some Python script to translate this into a notification for me that one of my packages needs an update, for instance, a bug assigned to me on the Chakra GitLab instance. I will publish this on Github when ready, alongside with setup instructions for the whole system, so other packagers can benefit from it.</p>
Sun, 26 Mar 2017 18:20:00 +0000
http://shainer.github.io/math/2017/03/26/game-theory.html
http://shainer.github.io/math/2017/03/26/game-theory.htmlmathThe packager manual (for Arch and derivatives)<p>I have poor memory. One thing I have especially poor memory for is Unix commands and parameters. On top of that, I don’t do
packaging often, so some less common commands escape my mind quickly. This
post tries to address this problem by listing some of the commands I find both
useful and hard to commit to memory.</p>
<p>It follows that this post is likely more useful for me than for anybody reading
this, but you never know. If there are other commands you use during packaging
that you would like me to include, feel free to let me know.</p>
<p>The reason I specify “Arch and derivates” in the title is because some of the commands are for <code class="highlighter-rouge">makepkg</code>. The rest, however,
are more generic.</p>
<h2 id="makepkg">Makepkg</h2>
<h3 id="only-download-and-extract-sources">Only download and extract sources</h3>
<div class="highlighter-rouge"><pre class="highlight"><code>makepkg -o
</code></pre>
</div>
<p>This also runs the prepare function, if present.</p>
<h3 id="do-not-download-sources-use-existing-ones">Do not download sources, use existing ones</h3>
<div class="highlighter-rouge"><pre class="highlight"><code>makepkg -e
</code></pre>
</div>
<h3 id="repackage-only-no-build">Repackage only, no build</h3>
<div class="highlighter-rouge"><pre class="highlight"><code>makepkg -R
</code></pre>
</div>
<h3 id="cleanup-after-build">Cleanup after build</h3>
<div class="highlighter-rouge"><pre class="highlight"><code>makepkg -c
</code></pre>
</div>
<h3 id="cleanup-before-build">Cleanup before build</h3>
<div class="highlighter-rouge"><pre class="highlight"><code>makepkg -C
</code></pre>
</div>
<h2 id="diffs-and-patches">Diffs and patches</h2>
<h3 id="creating-a-patch-from-two-files">Creating a patch from two files</h3>
<p>Make sure you execute this from the root of the source directory, so that
file paths are complete.</p>
<div class="highlighter-rouge"><pre class="highlight"><code><span class="gh">diff -Naur old_file new_file > my_patch.patch
</span></code></pre>
</div>
<h3 id="applying-a-patch">Applying a patch</h3>
<p>Make sure the cwd is set to the root of the source directory when this is
called:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>patch -N -i my_patch.patch
</code></pre>
</div>
<p>The <code class="highlighter-rouge">-pN</code> flag is useful to strip N parent directories from the paths
the patch refers to, in case you are located at a deeper path than the root.</p>
Sun, 26 Feb 2017 14:40:00 +0000
http://shainer.github.io/chakra/2017/02/26/the-packager-manual.html
http://shainer.github.io/chakra/2017/02/26/the-packager-manual.htmlchakra