What I mean by that is that it's in the same weightclass of speed, depending on the problem being tackled. In the case where data's shape is mutable, SBCL will scream ahead thanks to CLOS. You can cheat LuaJIT with dynamically-defined C-structures via abuse of the FFI lib instead of native tables, but it's not as nice as CLOS nor is it very safe. In the case that the shape of data is changed extremely frequently? CLOS might actually end up falling behind here. Another area where SBCL will likely win out is when the hotpath is bottlenecked on string operations.
Where I'd say it advances into breaking SBCL's jaw is that the runtime, interpreter, jitter, etc. are all much smaller than SBCL's runtime and compiler. If you're looking for a complete system, I'd say SBCL wins out obviously. You're talking a world-class REPL, debugger, a high quality stdlib, etc. All it's missing is a text editor like LispWorks (emacs and pretty much every other FOSS Lisp editor I've seen is a massive downgrade.) With that in mind, SBCL is not something you embed in an application written in another language. The holy grail is getting something as fast as SBCL, as flexible as SBCL, but as a 50k loc self-contained runtime. LuaJIT is the reigning heavyweight champ there, so having a Lisp-adjacent language like Fennel running atop it is a pretty damn compelling idea.
Interestingly with regards to text editors, Lua doesn't have that problem technically. Lite-XL is dangerously close to being zmacs/LispWorks for Lua. Poetically, just like Lua it's fairly bare bones and requires extension to be a decent IDE. But the underlying structure is absolutely fantastic, being based around a fairly cohesive object model rather than coats of paint over text buffers.
A small but illustrative example: given a live running clisp program, I can click on a UI element to inspect both the live object and the underlying code in the IDE. I can even copy and paste UI elements!
Emacs is text-centric, LispWorks is clisp-centric with a real GUI designed for computers with a keyboard and mouse. There's a whole MVC system! Wow!
GNU Emacs, Lem, etc. are all very nice editors, but they're stuck in the 1970s and are entirely keyboard-centric with very primitive UI flows. You can get used to it, but never fix it. If I wanted to use slime-fancy to inspect a class, what I get is a dead list of raw text that I have to awkwardly interact with. When I inspect a class in LispWorks, I get truly interactive UI elements which themselves map to rich Lisp objects.
Even editors like Lem that fix Emacs' performance issues (another genuine problem), they all make the same mistake of being centered around text-buffers and under-engineered UI frameworks. You can extend them ad infinitum, but if the roots are rotten...
Second Climacs exists, but it's more of a curiosity and a research project than a usable text editor IME (and is also currently tied to X, though that doesn't affect me personally). I hope that changes some day.
> A small but illustrative example: given a live running clisp program, I can click on a UI element to inspect both the live object and the underlying code in the IDE. I can even copy and paste UI elements!
I agree GUI is awkward in Emacs/SLIME. I think the reason is there isn't a standardized GUI framework across the Lisp world. Otherwise people can make SLIME support it.
> If I wanted to use slime-fancy to inspect a class, what I get is a dead list of raw text that I have to awkwardly interact with.
This part I do not agree. Nowadays "text" in Emacs/SLIME is far from "raw text", there's button that respond to hover, with right click context menu, and can be copy and pasted. I recall some small quirks (like objects in inspector are not presentations) but the only reason they aren't fixed (yet) is nobody get bothered enough. There's rarely any fundamental limitation. After all, it wouldn't be fare to call LispWorks UI "dead pixels".
BTW I once wrote yet another over ambitious project < https://github.com/neomacs-project/neomacs > but it doesn't go anywhere either. People do not care enough, Emacs is good enough.
I applaud the project, and I completely agree that the concepts maps nicely to SQL. The R equivalent of a WITH data prep block followed by the VISUALIZE is pretty much how all my plotting code is structured.
However, I don't see what the benefits of this are (other than having a simple DSL, but that creates the yet another DSL problme) over ggplot2. What do I gain by using this over ggplot2 in R?
The only problem, and the only reason I ever leave ggplot2 for visualizations, is how difficult it is to do anything "non-standard" that hasn't already had a geom created in the ggplot ecosystem. When you want to do something "different" it's way easier to drop into the primitive drawing operations of whatever you're using than it is to try to write the ggplot-friendly adapter.
Even wrapping common "partial specificiations" as a function (which should "just work" imo) is difficult depending on whether you're trying to wrap something that composes with the rest of the spec via `+` or via pipe (`|>`, the operator formerly known as `%>%`)
We are not out to convince anyone to switch from ggplot2 (and we are not planning to stop developing that).
ggsql is (partly) about reaching new audiences and putting powerful visualisation in new places. If you live in R most of the time I wouldn't expect you to be the prime audience for this (though you may have fun exploring it since it contains some pretty interesting things ggplot2 doesn't have)
Totally get that. I was mostly just long-windedly complaining that the one problem I have with it seems to be exacerbated by, not fixed, by this. I was also hoping someone would say “oh it’s actually way easier than you think, see (amazing link).”
I really do think it’s a good idea to explore! Sometimes I feel crazy because I’m the only one in my department that prefers to just write SQL to deal with our DBs instead of fiddling with a python/R connector that always has its own quirks.
Side comment: |> and %>% aren't the same btw! The newish base pipe (|>) is faster but doesn't support using the dot (.) placeholder to pipe into something other than the first argument of a function, which can sometimes make things a little cleaner.
The base pipe has an underscore as a placeholder. From the docs:
Usage:
lhs |> rhs
Arguments:
lhs: expression producing a value.
rhs: a call expression.
Details:
[...]
It is also possible to use a named argument with the placeholder
‘_’ in the ‘rhs’ call to specify where the ‘lhs’ is to be
inserted. The placeholder can only appear once on the ‘rhs’.
I believe this wasn't added in the initial implementation of the base pipe so some didn't realize it got included later, and still does not let you use constructs like e.g. combining multiple transformations of the input on the rhs. But for most purposes it's certainly sufficient
Most of the memories I have of my grandmother are post-serious Parkinson's progression. She was able to live a very long life, but it seriously affected her. Good luck, hoping for the best for you.
I’m not sure what your professional experience is in, but as a counterpoint, I’ve never been in a situation where I hadn’t wished for a system I’m working with to already be in a Bayesian framework. Having said that, I only occasionally am building things from scratch instead of modifying existing systems, so I’m not always lucky enough to be able to work with them.
The pain points around getting a sampler/model pairing working in a reasonable timeframe is definitely a valid complaint. In my experience, inference methods in Bayesian stats are much less forgiving of poorly specified models (or said another way, don’t let you get away with ignoring important structural components of the phenomena of interest). A poorly performing model (in terms of sampler speed/mixing) is often a sign of a problem with the geometry of the parameter space. Frustratingly this can sometimes be a result of conceptually equivalent, but computationally different parameterizations (e.g. centered vs non-centered multi level effects).
The struggles are worth it IMO because it is helpful feedback that helps guide design, and the ease with which I can compute meaningful uncertainty bounds on pretty much any quantity of interest is invaluable.
My go to for teaching statistics is Statistical Rethinking. It’s basically a course in how to actually thing about modeling: what you’re really looking for is analyzing a hypothesis, and a model may be consistent with a number of hypotheses, figuring out what hypotheses any given model implies is the hard/fun part, and this book teaches you that. The only drawback is that it’s not free. (Although there are excellent lectures by the author available for free on YouTube. These are worth watching even if you don’t get the book.)
I also recommend Gelman’s (one of the authors of the linked book) Regression and Other Stories as a more approachable text for this content.
Think Bayes and Bayesian Methods for Hackers are introductory books from a beginner coming from a programming background.
If you want something more from the ML world that heavily emphasizes the benefits of probabilistic (Bayesian) methods, I highly recommend Kevin Murphy’s Probabilistic Machine Learning. I have only read the first edition before he split it into two volumes and expanded it, but I’ve only heard good things about the new volumes too.
Any time you start conditioning on something, i.e. selecting subsets of data to analyze. You can fool yourself quite often if you do something seemingly innocuous like select "everyone with at least one X" and compare expectations to what's true unconditionally (meaning not conditioning on anything, not "in all cases") with conditional computations.
Agreed wholeheartedly. I have argued with the VP of our department about this paper quite a few times.
I feel like Breiman sets up a strawman that I've never encountered when I work with my colleagues that are trained in the statistics community. That doesn't mean it didn't exist 25 years ago when he wrote it. I concede that we are sometimes willing to make simplifying assumptions in order to state something particular, but it's almost like we've been culturally conditioned to steep everything we say with every caveat possible.
Whereas I am constantly having to point out the poor feedback we've had about some of the XGBoost models despite the fact that they're clearly the most "predictive" when evaluated naively.
What exactly do you mean by this? Speed? Portability? Ease of use?
reply