Years ago I got my first internet enabled mobile, and the carrier advertised it as having "300 free minutes" as part of the trial period, which is great. So I used 297 minutes of internet services in the first month, but aha actually the minutes only referred to telephony, and I was stung with a ~12,000 dollar overdue mobile internet invoice (360 dollars per megabyte or something equally ludicrous)
They got done in by a massive class action, that I was tangentially a beneficiary of, not because of the minutes claim, which was standard practice, but because they had failed to provide anyone with the cost of the data.
I think I paid them 300 bucks or something in the end. After further letting a 600 dollar agreement go to collections and settling with collections for 50%.
Had a tiny (<400) overdraft covered just minutes later. Called bank, they said the payment causing the overdraft wouldn't post for 24 hours. Don't remember if that was policy, or something the person would do for us.
Several months later, I learned at that time, that they had instituted a standing stop payment order to the merchant we were paying, but only in the amount of exactly $600. Fast forward, we make a $600 payment, get a notification that it went through, no notification of stop payment, but later a notification from the merchant we were trying to pay (Synchrony - Amazon Store Card) stating they were closing our account due to the stop payment.
Meet in person with a bank rep. They have trouble finding the problem, but once found, argue that we must have requested this specifically, for they would never have done such a thing. I didn't even know a standing stop payment for a specific amount was a thing, asked what is normal policy, what bank terminonlogy would cause such a thing. Would not give any answer other than "you must have asked for it". What would I have said? "You must have asked for it" etc.
Currently fighting with the bank for audio of the phone call where they claim we requested this, and with Synchrony telling them it is the bank's fault. We immediately paid balance on closed Synchrony account, and had made ~12 payments successfully before the inadvertent stop payment.
In the US this always seems to be called "overdraft protection", as in "do you want overdraft protection on your account?"
I can never remember if overdraft protection means "we won't let your account overdraft", or "we will let your account overdraft"
It is an incredibly confusing name to me
I could look up the answer now, but I know I will just forget it again. I must have looked it up and then forgotten at least 10 separate times over the years
Normally means that if your checking accout balance hits zero, they will pull from your savings account rather than returning a check presented for payment. They may (probably will) still charge you a fee when this happens, but it might be less than an overdraft.
Or, it might be a case where they grant you a short-term personal loan to cover the overdraft (up to some limit) rather than return checks. Again there will be fees and interest on this.
In either case, since they are not rejecting payments, you avoid getting hit with fees from whomever you wrote the check to. So your only fees/penalties are paid to the bank.
Of course the best answer is just keep better track of your checking account balance.
Of course, the funny part about that is that they charge you to check your balance at the ATM. It's almost as if they don't want you to and have to pay their fee.
Which of course is about as odorous fee as possible. You don't have enough money, so we're going to charge you more money?
It probably depends on the bank, but setting one's overdraft limit to zero doesn't necessarily help and may actually make things worse: some payments may still go through even if you don't have the funds, putting you in unauthorised overdraft, which tends to have a high daily fee.
When one of my sisters got her first paycheck from her first real job, she didn't knew much about banking, and thought that the overdraft limit on the ATM was her actual salary, so she just withdrew the whole thing. I think it took over a year, maybe two, to pay it all back.
For consumer products, definitely. Any unreasonably large charge (with a comfortable margin, like 10x) should be waived by default, unless the client specifically requested the limit be removed.
I love hard caps, and am tired of cloud services not even offering those. May make sense for large companies. Makes no sense for hobbyists and small companies. But maybe that's the point?
Many many years ago my girlfriend at the time was on holiday with her mother so I'd added a "Roaming Bundle" to her phone.
While she was away I got a call from our phone provider that she'd racked up £1700 of roaming charges. "But don't worry about it!" said the customer service person, "Just stop your direct debit for this month just in case and we will call you when the bill is generated."
A couple of weeks later, they rang back, confirmed that the final amount was about two grand, and so I said "hang on, there was supposed to be a message when the bundle was halfway used and a message when the bundle was nearly used up - what happened to that?"
"Oh I don't know why you'd have been told that, we've never done that! But you're right, that's what they said, we've already listened to the call. What you *should* have been offered was this *other* bundle which would have cost £150, she's used about 75% of the data that would provide, plus the £25 you originally paid for the first bundle, call it 75 quid then? I'll just take the card number..."
And that, gentle reader, is why I'm still a customer.
Last century most people were only sometimes using the Internet. They'd "Go online", if you're young enough not to recognise that noise in Blue Prince when you use the network, that's a "Dial up modem" which is how a typical person would "Go online" in the mid-1990s. So in this regime counting minutes makes some sense.
I got a preview of the modern world from about 1996 because my first shared student house (shout out to any Hitchers reading) had a single dial-up modem set to always connect to a free-to-use University modem and then used IP Masquerading (the ancestor of today's NATs) so that all of our computers could share this tiny connection.
So by the time of my 21st birthday, I was "always on" in the same sense that you'd always be today, except with much, much lower bandwidth and what I can tell you is that this, not the bandwidth is what makes the difference.
When you're "always on" the reflexive answer to "Wait, where do Porcupines live?" is to look online. It's 1996 so Wikipedia doesn't exist yet. Google doesn't exist yet. But Tim's crap hypermedia system (the "World Wide Web") exists and so you just need to know where to look to find information about porcupines.
I didn't watch videos in 1996 because it'd take hours to receive a short low resolution video, even reading web comics was quite an undertaking, I remember downloading all of Bruno (at the time) https://en.wikipedia.org/wiki/Bruno_(webcomic) over night so I could read it. But the fact you don't have to explicitly "go online" makes a huge qualitative difference even though the bandwidth is tiny.
> if you're young enough not to recognise that noise in Blue Prince when you use the network, that's a "Dial up modem" which is how a typical person would "Go online" in the mid-1990s
I have some lovely high-quality video of dialup modems both dialling out and dialling into with good audio, which I'll post up at some point.
We still use them at work, at least until BT finally cut the copper services off at the end of the year.
The similarity between mobile internet (CSD) and dial-up wasn't really obvious from the users' perspective because they weren't explicitly making a call to access the internet. The session was established transparently when data had to be transferred, and the time this took was charged as minutes.
Operators always dreamed of a world built on circuit-switched networks that they fully control instead of packet-switched IP networks where anyone can take part and operators are just a carrier. So the big operators started the mobile internet era with the telephony model.
Circuit switching has latency and QoS advantages, but it's definitely not worth the cost for most use cases.
Interestingly, TDM carriers have a latency of approximately one bit per hop and jitter less than that, while any kind of packetized voice such as VoIP has a latency of at least one packet worth of audio, plus one to several packet transmission times per hop, and jitter of several packet transmission times, and packet drop on top of all that. We've actually made quality worse in the name of fitting more services down the same wire, which isn't necessarily bad but it's something to think about.
Things I learned today: the very first mobile (cellular) data protocol was indeed dial-up over the GSM voice line: namely the CSD. What I had in mind - GPRS - came only later... so yes, even on mobile it was initially billed by minute. Wow.
In the early 2G GSM days, you would connect via "CSD" (Circuit-Switched Data[0]) which essentially emulated a dial-up connection (except it was digital the whole way) and that was billed per minute. Using HSCSD you could bundle multiple CSD channels for higher bandwidth but the minute charges were per channel so the cost added up.
By the time phones got built-in WAP browsers, GPRS had arrived which was billed per byte, but CSD was still supported (I remember using it once or twice in a certain building at uni where GPRS never worked for some reason)
Very early on, mobile data was essentially an emulation of an analogue modem call, at blazing fast rates like 19200baud or less, held over the same channels as voice calls with a bit of extra jiggery-pokery, and were charged by the minutes just like voice calls. Sometimes they were charged as normal voice calls, sometimes you ended up being charged extra (or not being charged extra as such, but with data calls not being counted in any pre-paid minute allowance your plan had). It was vitally important to check this before making modem calls, as some providers charged data calls in the same band as premium rate voice calls, meaning that just picking up a few text emails could be quite expensive.
[cue misty eyed memories of trying to get my Psion 5mx to connect to the Internet by infra-red serial link to my phone, to try get updated connection information for my next train as the one I was on was going to be too late to make the planned connection…]
Other way around. The phone plans were billed by the minute. Then came the Internet. In is first generation, Internet was essentially just a long distance call through a modem. Hence it was billed - like the call - by the minute.
Dedicated Internet wires came much later, and then the dedicated phone lines were dropped as voip was better quality and cheaper compared to the dedicated lines.
While the phone still had a dedicated line it didn't actually need a power connection, as the power through the phone wire was sufficient.
I'm thoroughly confused as to how what you're saying differs from my previous post, but I believe you just misunderstand that by "early phone plans" in my post's and the parent post's context, I meant phone plans for Internet access ("early" being the feature phone era).
Online services most assuredly billed by the minute. He'll, AOL had a huge marketing campaign offering "free" minutes for new customers. You might also have charges from the phone company but those were independent of the online service charges. It wasn't until the late 90s the major online services went to flat rate "unlimited" plans.
> Online services most assuredly billed by the minute. He'll, AOL had a huge marketing campaign offering "free" minutes for new customers. You might also have charges from the phone company but those were independent of the online service charges.
AOL was offering free minutes because it was an ISP, not because it was an online service. (It was also an online service. Most of that service was indeed free; some of it billed by the minute, but that was separate from the rate you paid for connecting to the internet.)
> Dedicated Internet wires came much later, and then the dedicated phone lines were dropped as voip was better quality and cheaper compared to the dedicated lines.
The telephone network made the utterly bizarre choice to intentionally degrade the audio signal of a call, guaranteeing that people would have an unnatural, distorted voice if you spoke to them over the phone. There was no way for voip not to be better quality.
If you were used to dial-up internet, (IE, where your computer was plugged into a phone line and made a phone call to your internet service provider,) that's a reasonable assumption to make. Some early services billed by the minute, too.
WoW. That's certainly a surprise to me. I'd never expect an invoice after not putting in a card.
I also believe this is totally just a case of "billing and metering is hard, and may actually be a larger engineering effort than your actual service".
I was just looking at them earlier today since our Github actions are slow AF, and while they sounds great, this tells me it'll cost me more time to make sure I babysit it than most other trials.
With most of these, they end, the service stops working, and you have a choice to make: (a) it was worth it sign up, (b) not worth it revert.
Founder of Depot [0] here. Feel free to try us out. We have a real free trial that is time based that doesn’t do odd things like this. Also have usage limits that you can put in place to further clamp down on runaway surprises.
A certain number of businesses will just pay an invoice if it looks real and has a business justification. There are scams around this, but also vendors who may be taking advantage of this. If 30% of customers will just start paying the invoices, it's worth it to them to deal with the questions from the rest of them. At least until reputational damage starts.
A very simple example of a scam that's probably happened to you if you own a domain name: as the expiration date approaches you'll get "invoices" from companies for domain name renewal charges. If you read the fine print it will say "this is a solicition for business" but otherwise it looks just like an invoice. Some people will just pay them.
Many years ago I met a woman that ran a "business" that simply invoiced legitimate companies for printer supplies and toner every month. A surprising number of her invoices were paid without question.
That went on for a couple years. Then one day she left town basically in the middle of the night and we never heard from her again.
This reminds me of the business practices of the Austrian NIC. Usually domain names expire if you don't renew them. In Austria, unless you explicitly cancel the domain name by fax, they just roll the registration over to the next year and then send you to collections[1] if you don't pay up.
There's no rule that domain names expire unless you renew them, at least for ccTLDs. It's just a convention. Conventions lead to assumptions, and assumptions can be used to scam people.
In general there's two types of businesses: businesses where you pre-pay (e.g. McDonalds), and businesses where you post-pay (e.g. a sit-down restaurant). If you take a conventionally pre-pay service and apply post-pay pricing to it, you have yourself a perfect scam.
> I received a letter from the debt collection agency. What can I do?
> If a domain hasn't been paid for despite several payment reminders from nic.at, the domain shall be locked and the open claim handed over to our debt collection agency. As a result, the invoice must be paid directly to the debt collection agency. Please contact our debt collection agency for more information
Like TFA it's hard to tell if they genuinely believe that they are helping their customers by not discontinuing their service, or if it's a scam. I suspect a mixture of both.
It's both. It's normal business practice in the German-speaking countries (a.k.a. DACH) that would be considered a scam anywhere else.
Basically, in most countries paying money is something that requires continuous enthusiastic consent - if you don't pay, that's the business's problem and they should stop serving you, and they may only recover payment for goods they've already given you and not received payment for. But in DACH, it only requires technical consent - if you signed something saying you'll give them money, then you have to give them that money, and you cannot rescind your obligation to pay, except as provided in the contract or an overriding law.
You went to Austria and did Austrian business with an Austrian company, you should be aware that Austrian rules and norms apply. ccTLDs are not generic, every country is free to apply any rules on their ccTLD!
> You went to Austria and did Austrian business with an Austrian company, you should be aware that Austrian rules and norms apply.
The consumer purchases a domain name from their registrar. Neither the consumer nor their registrar are in Austria. The registrar is the one providing the service of registering the domain with the NIC. The NIC can't just say "we have a contract directly with the consumer".
The registrar can write in their contract that "consumer must cancel domain before it expires with the NIC" or "consumer must sign contract with NIC", and if the consumer doesn't do that then the registrar can maybe (depending on consumer and contract law in their jurisdictions) sue the consumer if they don't fulfil the contract of sale. But writing in one contract that the consumer must sign a third-party contract doesn't imply that the third-party contract actually exists. The NIC cannot go after the consumer because they don't have any contract with the consumer.
This creates some very uncomfortable cultural clashes sometimes. I remember way back when I was signed up for a service provided by Ms. They did not, as far as I could tell, provide a way to cancel the service, other than to stop paying. Since I had provided them with automatic payments, my bank did not allow me to simply stop paying their invoices, I had to cancel with the provider. I talked to my banks customer service, they refused to help. So I cancelled my account with that bank and moved all the money to a different bank.
Rather more complicated to cancel that service than I would've wished, but hey, I got a better banking service out of the deal, so it's not all bad I guess
Being unable to tell how to cancel something is a different thing. You can contact their support. If they don't have support, or the support refuses, then you contact your bank, with evidence, because it's now an unauthorized payment.
This is separate from the cultural norm that you can only cancel with a certain notice period or by fax.
Yes? ICANN is part of the US, and has been strong-armed by the US government into imposing restrictive rules on gTLDs even though they appear like they're global.
Any actual evidence that the US government has control over ICANN? I know they used to have an agreement that gave the US some oversight, but as far as I know that ended a decade ago.
That may be how things work in some countries, but not in the US, even under Trump.
Let’s Encrypt evidently didn’t mean to block all of Iran, only the Iranian government, which I assume is capable of getting its own certificates: https://news.ycombinator.com/item?id=48465754
> In order to use the Blacksmith Software Inc Service, You must set up an account. During the account setup process, You will be required to connect your GitHub account and install Blacksmith’s GitHub integration in your org, and add a valid payment method, such as a credit card, which will be processed through Stripe. Alternatively, for larger contracts, You may request to be billed via invoice.
> By providing payment information, You authorize us to charge Your credit card for usage fees or, in the case of invoice-based contracts, agree to make timely payments as specified in the invoicing terms.
Unless this guy had a larger contract and requested to be billed via invoice, this is a violation of terms and he should tell them to stuff it.
This doesn't seem like the right way to do business long terms. The off chance that someone actually take you up on it and pay your 'bill', you've destroyed a lot of goodwill and alerted the rest of the tech world of your scammy moves.
Indeed. Have never used Blacksmith and maybe there’s a good reason why they do this, but it seems odd that in world where so much of the signup and onboarding process is fine tuned for optimal conversion, a business would take an approach to a free trial that risks leaving such a sour taste in a prospect’s mouth.
Also for those that require a credit card for a free trial, I always use a virtual card and cancel it. It's super fun to watch them cry when they can't actually charge you.
Funny, I got a fraud call recently because CrunchyRoll decided to try to renew a subscription I abandoned years ago and the card they have is expired.
I know it wasn't me because I gave up entirely on the service after they changed something about their login systems to reject my password and I could no longer get in. Support wanted me to jump through a lot of hoops and I just refused, choosing instead to just stop doing business there because I wasn't really watching anything at that point anyway.
This was around 2022, mind you, so they tried to renew me after several years with no explanation.
This happens with me every year with Barnes and Noble bookstore. I canceled their annual membership back in like 2018, Starting in 2021 this zombie account started charging me for a renewal. I reached out to support and they disavowed any knowledge of how this could happen, had no record of charging me, and no clue how it could have happened.
Since Amex is nice about allowing someone who charged you in the past - keep charging you even as your card has changed, they allow the charge.
Every year I do a chargeback, every year. Amex can't figure out how to not allow it and B&N has no idea why they charge it. Hmmm...
It's way easier to just not give them a way they can charge you. That way you don't have to deal with a support representative fakely asking you how your weekend was, and who doesn't actually care about your weekend.
$1000 in a month? What on earth are you doing in CI? We're on Warp build (50% cheaper instead of 33%) with 6 people hitting it pretty hard across a few repos and only reach $150/mo. That's with building rust - a worse case than anything you're using for build time.
1. I'm not sure if blacksmith offers larger runners, but you if you're using them you have checked that bigger runners are worth the squeeze? A 2x runner does not mean a 2x faster build - I had a goal and sized CI according to that.
2. Caching. Nx can do this for your TS, provided your code is decomposed into packages.
3. At a previous job I had a post-run timings job. GH markdown supports Mermaid; I used a gannt chart to express it. I don't remember if the GH api supports getting timing for the current wf - so it may have been a second workflow.
The first is just a little legwork, the rest your agent should be able to do in 5 minutes.
Having been a happy customer of Blacksmith, Depot, and Ubicloud at various points (all three of which offer GitHub Actions runners at cheaper prices than GitHub themselves), I agree the billing here is weird. It's worth pointing out, though, that you have to use a lot of CI minutes to incur a $1000 bill. You're well outside of hobbyist levels, and that same level of usage would have previously cost double from GitHub themselves. This a business with serious compute needs, being asked to pay real money.
Yes, having a performant CI server on premises would probably pay itself in less than three months.
Startups are usually resource-rich and time-poor, and don't want to allocate employee time to infrastructure unless it's core to their business, so arguments for efficient spending there are easily overlooked.
We used to have self-hosted runners on Hetzner dedicated servers, and we migrated to UbiCloud ~6 months ago.
Unless you can optimise your CI runtime to be very balanced and fast, you're going to have a lot of jobs queued during working hours ... although it costs more per minute, the fact that CI jobs are bursty means it works out not too much costlier but with less queue time.
This reminded me of our experience with Gusto - we signed up to their R&D credit payroll offset service in May last year - their offset fee is calculated as a percentage of the benefit you get. We filed our federal tax return in September, so there naturally wasn't any payroll offsetting until October. They still charged for services they didn't deliver so it was more cash out on day one which really goes against the purpose of the tool. They argued it was in the small print that said "we charge when you sign up" when I ticked a box. I thought that was quite outrageous. I still do.
I would test https://avrea.com
They are from Helsinki, just launched and the founding team has multiple SaaS businesses behind them. Depending on the workload it could be really fast.
And they are from Helsinki so big plus for anyone from Nordics.
> This question is for us: will we keep using Blacksmith, despite them giving us an unpleasant surprise and a prickly support exchange?
Interesting question, and I think it makes sense they’ve chosen to be pragmatic.
However, I wonder if Blacksmith get bitten in the arse by this. Hopefully that changes their behaviour but, like many startups, they might simply fail.
So I suppose the trick becomes to keep using the service without getting locked into it until it becomes clearer whether they will succeed or not, then perhaps you can consider taking advantage of platform features.
Even then, I don’t know how much I trust Blacksmith or would want to make it hard for myself to move away from them.
And on GitHub actions: Microsoft are very good at owning the platform and then making products and features that are just useful enough that it’s not worth switching to a better alternative but absolutely no more. GitHub Actions is an obvious example, Teams is another, but the list is long. To me it reads as a more modern variant of the anticompetitive behaviour of the 1990s. It sets up enough of a barrier to keep others out, and kills innovation. I’m not a fan.
I am sure Blacksmith is the easy path if you don't want to do any extra work and I am also sure their solution will scale much better as this is what they do full-time.
However, I can share what we did to ease on our GitHub Actions bills if it helps.
Effectively we have our own runners hooked so that a job is scheduled, a runner picks it up and goes with it. We still use GitHub Actions but our monthly bill is now flat because we pay for a server. It is about 6x cheaper if not more.
The solution is not open source but it boils down to a Go service that orchestrates firecracker vms. All the vms are pre-warmed so there is always a fresh supply of workers to pick jobs of various sizes.
It is basic and it works. We have not had any issues since deployed.
The runners can be anything from 1 cpu 2 GB to 64GB 8 cpus. We can add more worker types in a config file.
I am not exaggerating when I say that we used to pay 1000s per month for this. Now our bills are in the range of a few hundred. Other dev boxes are done in the same way.
> We still use GitHub Actions but our monthly bill is now flat because we pay for a server.
Caveat here being that GitHub is exploring charging a usage-based fee for self-hosted GitHub Action runners [1]. While they've halted it for now, it's something worth being aware of as you assess your costs. This is probably a drop in the bucket compared to the order of magnitude savings you've described.
Allegedly this is because whenever you have a self-hosted runner, it's actually a Microsoft-hosted runner that is notifying your runner and then polling for it to complete. I wish I was joking.
Haha, we don't use Kubernetes. It is a single large server with plenty of space for runners. It is as simple and boring as it gets. The only fun part is that we use Firecracker for the actual runners. It is provides nice isolation and it is very easy to work with.
As someone who has worked with gitlab for the last 5 years, i find it amazing how over complicated the github flow it for CICD! Spin up a server and add the gitlab agent and done! You need more? Fine spin up a EKS k8 cluster and done!
You can do the same with GitHub Actions. Spin up a server and start the actions runner on it and done! But if you want to dynamically scale your runners in response to load using a Kubernetes cluster, then they provide Action Runner Controller and it's a bit more complicated.
I can see their logic - instead of breaking people's builds, they are being the nice guys and letting you pay them back later. BUT, any time you break convention, you've dipped into your trust budget (even if you communicate it way more clearly than was apparently done here.)
> While amusingly as of June 8 Blacksmith’s terms implied that their right to bill you is contingent on you providing payment information, a SaaS app certainly could have terms that obligate users to pay for unexpected overage when on a free trial.
> And let’s be clear: our agents run a lot of CI jobs, so we did expect to hit the limits of the free plan. We used the service and got value for it. So it’s not inherently dishonest, just surprising. My read is that they can do this.
So I read this that the terms say "blacksmith will only bill you if you provide payment information" but then they say that blacksmith can bill you if you don't provide payment information. That seems to contradict the terms, which I would assume underlie the "contract" that you agree to when agreeing to the terms.
Hey folks - Greg here, I do product at Blacksmith.
I want to say upfront - we've never pursued these invoices. If someone feels they didn't get value from the service, we've eaten that cost and always will.
There's a bit of an implicit policy decision on our side here that we did a bad job of communicating - I want to clarify that, and then talk about how we can fix it.
First, we wanted to let customers start using Blacksmith without a credit card. Very few infra startups do this today - CC validation is great for anti-abuse - but doing so has let us support a much greater number of free users.
Many of these free users have turned into full OSS sponsorships (most recently ccusage, and before that, OpenClaw) or large paying accounts, and we haven't wanted to cut those users off from trying us. It's a real pain point to find a credit card to put down for your company or OSS project's CI spend before you've even tried the service.
Second, not having a credit card on file means we don't actually know when credit card-less users intend to continue past their free trial.
Shutting down users CI workloads entirely seemed harsh, especially because doing so would fail builds and require a code change to resolve. If users could start without a credit card, we weren't going to then hold their runners hostage. Instead, we decided to just eat the cost for the small number of users who either abused our services or did not actually mean to use the service.
This worked, mostly - though every month we have gotten a number of support cases with users confused about their invoice. If they didn't intend to use the service past the free tier, we've voided out the invoice, and often given credits against a future bill if they intended to use the service, but were surprised by the behavior.
We have a lot to improve about our billing mechanics - but because our retention rate for these users has been so high, we have assumed great support could catch and resolve the ambiguity.
That said, there's two changes we can make now:
1. we clearly missed the mark on supporting this specific case - we should have offered to void this bill entirely given the surprise factor here.
2. We're prioritizing up making progress on a Wallet implementation that will let folks choose to suspend their runners rather than let them continue to run after they use up their free tier.
We also just launched a new billing/metrics view so users have better visibility into their free tier and Blacksmith usage.
I'm sorry for the bad taste this has left in everyone's mouth - I'll be hanging out here and on greg [at] blacksmith [dot] sh if you want to talk about your account specifically.
I appreciate you stepping in and clarifying. Thank you.
But honestly, this feels like damage control. This is (to me) clearly an innovation on a dark pattern that is basically just "accepted practice" nowadays, namely the "subscribe by default; make it hard to opt-out of said subscription at signup time".
That's why I think people are upset about this. By not taking a credit card, you made it feel like it wasn't an implementation of that dark pattern (yay!) -- however, secretly, it was!
If the intent is to be customer friendly, it's so perfectly clear to me what the answer is. OPT-IN. In other words, a checkbox:
"By default, when your free credits are consumed, all of your runners will be de-provisioned. Instead, if you would prefer that your runners continue working, check this box and we will invoice you for usage in excess of your free credits."
But honestly, you did more than I expect of most service providers today. You sent an email; you actually told them how many credits were left in the free tier. AWS, for example, can only give me an _estimate_ of how much cost I've accrued. They can make no promises about the rate at which I'm expected to accrue new costs. And, unless I've taken great care (by, say, terraforming every resource in a given AWS account), "turning off my cost accruing services" is not a simple matter. If I understood the article correctly, there was, at least, a "single action" they could have taken to immediately stop accruing costs.
Yeah, I hear you on the inversion of expectations and I think you're exactly right about how this affects perceptions of the product.
Candidly, right now, we're more likely to land on an opt-out rather than opt-in here since getting your builds to actually run again requires a code change. I think there's a full blog post to write about why we think this is the right default, but we're still thinking it through.
Regardless, we need to make the choice loud and visible, and our miss was making it too hidden. We'll close that gap.
> I want to say upfront - we've never pursued these invoices.
At first glance that sounds admirable, but the flipside is that it implies Blacksmith knows they're being shady: if you know you're not going to pursue it, why did you invoice the customer in the first place? This sounds less like you're forgiving a charity case and more like you're waiving an invoice because the customer identified it as unethical to begin with.
I partially disagree! We issue an invoice because the customer did consume the services - a lot of folks appreciate that, since it gives their AP team a real cost to work from instead of a ballpark of us vs. GitHub.
My "we never pursue these" comment wasn't about us knowing the charge was shady - it was to clarify we have no strategy of chasing people/litigating over invoices they weren't aware of, which came up a bunch in the thread.
I do agree that pushing surprised customers through a support ticket is wrong, and that our in-product language compounded that issue. Controlling this belongs in the product so we can hit the mark on trust - we're building that now.
This isn't shady for most folks, since they're typically comparing us to known, expensive alternative. For those who are surprised by their spend (which given GHA's upper-bound pricing has been a minority) our reconciliation flow was for them to contact support, and my comment was to clarify that we don't have an intentional strategy to pursue folks for invoices they weren't aware of. It's clear to me that controlling this behavior belongs in the product so we can hit the mark on trust - we're working on adding that now.
> Alternatively, for larger contracts, You may request to be billed via invoice.
> By providing payment information, You authorize us to charge Your credit card for usage fees or, in the case of invoice-based contracts, agree to make timely payments as specified in the invoicing terms.
And at the top there's a Try For Free button that says no credit card is required. This strongly communicates that this free trial won't incur any costs until you add a card or agree to be billed via invoice.
A simple change of the text would make people a lot less surprised. Warn them that if they go over they will be billed. In the bill clarify at the top that they don't actually have to pay if they don't want to.
Agreed - we've made this copy change to the free usage email now, and the change to add a toggle for preventing runs when you run out of credits is going through CI.
Do you plan to refund all previous free trial customers, who didn't contact the support, and actually paid unexpected overage fees?
Invoices are legally binding documents in many countries, and even if that might not be the case in your country, not everyone might be aware of that fact.
> since it gives their AP team a real cost to work from instead of a ballpark of us vs. GitHub
Same could be achieved by showing the real cost in the web app and/or sending a report via email, without scaring them away, and possibly _extorting_ them.
> I want to say upfront - we've never pursued these invoices
So those people who would feel guilty not paying are punished and those who prefer to try to cheat the system by skipping payment on invoices are systematically rewarded?
As a consumer, I can appreciate your focus on support. I get a monthly $0.00 AWS bill on an account I've lost access to. However, because I have no outstanding balance, support is unable to assist me (is their claim). So I enjoy one monthly spam email from them, which I neurotically read in case one month the balance for whatever reason exceeds 0. Still less stressful than trying to resolve the issue with them.
The main issue is that you deceptively communicated what happens at the end of the free trial. You decided to use the wording "disruption", which implies that workloads will be stopped, which is the opposite of what happened. This is deceptive and predatory. I will never use your software and after this happened I am AMAZED that anybody else (particularly OP, but also anybody who read this post) will continue to trust you with their workloads. The way I look at this is: if you're willing to do deceptive and predatory billing, why wouldn't you be also willing to sell access to intelligence agencies or some other "grey area" party who wants to do supply chain hacks?
This whole incident supplied them a lot more publicity. I had never heard of them before today. Since today, I'm more likely to use them, not less, because now I know they exist. I couldn't be less likely than zero.
Technically, here the paying customer experience was the problem, not the free trial one, because they were automatically upgraded.
Generally, you don't want to award people/companies that don't respect you. I'm not saying that you shouldn't re-evaluate them in the future, but they will walk away with the wrong lesson if you just start throwing money at them.
If you view it through a paying customer lens instead, then all you see is a paying customer who chose not to pay their invoice and might get sued. There's nothing strange about that.
All of the problem here was with the transition from free trial to paying customer.
Personally I do care if I get hacked or if my clients get hacked. If you don't care, you don't care, I guess. For you it's enough that the "paying customer experience" (???) is good, whatever that is. Okay buddy.
First off, fair point on the free tier email. This fits in the same theme of clarity vs. trust for me: if we say disruption, but then leave your account enabled, that still feels deceptive even if it seems generous to us. We will fix.
On your last point - I just want to note that this isn't a money-maker for us. Runner bills can be large (as they were here!), and we explicitly wanted to avoid actual charges to the user in favor of not collecting for a good amount of usage. Again, there's a clarity/trust tension there that you're right to call out, but the end goal is that more people can use cheaper, faster runners.
We earn that trust today by being extremely easy to use and delivering on what we promise, and we need to add clarity to that list. Thanks for the feedback.
Reminds me of the company Tado, who was testing to see if people would pay by making them think they would have to even though they didn't. https://www.youtube.com/watch?v=tfAchfFXghc
I'm not so sure that Blacksmith is making the right choice around how to handle this signup flow, but I have very little sympathy for the person in this post. $1000 in CI is a lot of usage.
Actually quite common, as the cost to fight them isn't worth the legal fees.
One must be extremely careful when signing off on something as a company representative.
One internal IP lawyer wanted a legal journal subscription, and left the tap running after they left the firm... that one cost $8k if I recall, as the journal sold the delinquent account to a collection agency. Took 3 weeks to verify it wasn't a scam, as the companies usually go quiet without the account number etc.
Some people are wired that way, and run their company on legal cons. Indeed, one doesn't want to have these people around your firm. =3
A couple caveats. Doesn't support EC2 Macs, so you can't use this to build iOS apps. (Understandably. EC2 Macs are a pain to support and can't reasonably be scaled elastically due to a combination of Apple licensing restrictions and Mac hardware being totally unsuitable to the datacenter.) Also, this is a EU company which may be a problem for US-based customers.
> Well, we tried switching back to GitHub Actions, and… yeah it still sucks. Blacksmith has grown explosively because it makes an increasingly frustrating bottleneck in the dev cycle faster.
Where's the "grown explosively" bit coming from? Google doesn't have growth news from 2026, only early 2025.
That's incredibly scummy. Article author estimates that "only" 5% of people would expect this outcome for a "try for free", "no credit card required" service, but I think that number is well below 1%.
Can't believe they continued using the service after this. I would refuse to pay (they have no legal basis to require payment, and their own terms of service seems to disagree with their behavior) and find a more ethical provider.
Only the no credit card part is surprising. Something like AWS will bill you for everything not within the free tier and expect YOU to keep track of what's in the free tier. But they ask for your CC upfront.
It's not illegal or even unethical to bill someone who hasn't given their CC, but it's definitely unexpected.
Thanks for the warning. I would have chosen to be pragmatic as well but with another vendor. While I applaud your generosity with trust mine would have been evaporated at the first invoice.
I can understand not wanting to kill someone's CI in the middle of active development, but then the product needs a very explicit consent step before converting free minutes into deb
Not just the weird billing, which has been annoying the last month I've been trying it, but they have had a couple incidents lately where I feel like I got ripped off because their performance was so spotty and yet I was getting billed for it. https://status.blacksmith.sh/
Infisical does this too. They don't make it clear that they charge you for projects and machine identities upfront, and then you get slammed with a $800 bill on your first month.
Ooff, this seems a bit sketchy. We recently looked at GA runner alternatives and decided on https://avrea.com. See if you have a better experience with them :)
This reminds me of OpenAI which allows you to overcommit on prepaid tokens and then tries to force you into paying overpaid charges. Only they really can’t. You can’t make someone pay for a service they didn’t agree to with billing that doesn’t exist. I wish OpenAI best of luck with their shenanigans
Avrea.com founder here, if you'd like to try an alternative that won't surprise you with the bill, I'd be happy to give you some more credits for a trial.
Unpleaseant experience and all, but sounds like complaining about not reading the full terms and then being shocked after the first bill, which is ok if you are buying a personal service, but that's a no-no in business.
The argument of "like many early startup do, we oversaw this and ignored that" doesn't really make it better.
Most countries have consumer protection laws, but they're only for consumers, due to the usual power imbalance. Power is expected to be balanced in B2B transactions - your lawyers spent as much time ensuring you weren't scammed as theirs spent trying to scam you - so there's no protection law about that. You get whatever the contract says you get.
Kinda sounds like the post I replied to was ok with companies scamming or using dark patterns on consumers. Even if they're better protected it's not like it's perfect.
Presumably they wanted feature parity with Github Actions. If you have lots of workflows already defined in Github Actions DSL, you're not going to want to port them. And even if you do port them, the activities available in the place you port them aren't going to be the same as Github Actions; there won't be feature parity.
While I think it’s outrageous to send invoices to folks who aren’t paying customers… I’m a little surprised the service even has a niche (I’m old). I guess, with ubiquity of containers in our modern workflows, it seems strange to pay a service for what I assume is a dedicated, or well provisioned VPS, just to run CI. Hell you can probably get Jenkins (further showing my age here) running in less than an hour with Claude, GPT, or Deepseek on an obscure provider that offers cheap bare metal instances.
And for anyone who hasn’t used bare metal instead of over provisioned VPS for services the performance gap is noteworthy and substantial. Yeah, there is some risk because you have to worry about outages, upgrades, and configuration but for something like CI where there’s near zero data loss risk… it seems well worth it if performance of your CI/CD infrastructure is really an issue.
They could if they feel it's worthwhile. Most companies don't, but most companies don't do most of the stuff mentioned in this article, because they're lazy. If they're not lazy they can absolutely follow up any unpaid debt in court, no matter whether you tried to use a virtual credit card or anything like that.
In German-speaking (DACH) countries the companies aren't lazy and they will take you to court and the court will make you pay all legal and court fees as well as the debt. It's a near certainty they will bother. In the USA you're hoping they won't bother and they'll be satisfied with just banning you as a customer. I think this is because each party pays their own legal fees in the USA.
This is another level up (down?) from the standard "gotcha, b####!" post-usage metered cloud service billing that requires a credit card up front which is the main business model for a surprising number of internet companies. They can keep doing it because a significant percentage of their business is from startups that are burning through someone else's money.
But it's interesting that the only real difference is that they let you proceed without putting your credit card info in.
So ive done this before when I deal with predatory and 'free trial' companies.
Go get a $20 gas station credit card. They get THAT card, and whatever name you want to provide.
When they demand $x000 for their free trial, they get.... $20!
And before anyone bemoans their 50+ page onesided "contract" that weasel-words revokes 'free trial'... Sure, they can publish a claim and sell off a debt to "John Q Public". We can see how far they'll get with that.
If they can prove you agreed to the charges, they can follow it up with debt collection companies or lawsuits, and that will cost you a lot more than just paying what you agreed to pay.
And let me know how a lawsuit against "John Q Public" will go, along with an email from fakemail.net and a gas station preloaded credit card.
Companies deserve this sort of treatment if they say shit like "FREE TRIAL" and silently convert you to 'you owe money' with no hard limits. And naturally, paragraph 37, sentence 12 includes this disclaimer. That should be blatantly clear, but its not cause they are scammers.
Oh, that's fun, because IIRC providing fake details makes it criminal fraud, automatically. They get to seek security camera footage from the gas station, subpoena your ISP to find who held your IP address (hope you used Tor) and subpoena fakemail.net for the same.
Holy shit, all that and then they paid? Worse, decided to use this product on an ongoing basis? I'm not sure who I hate more, Blacksmith for doing this or OP for being such a doormat about it.
That's a typical discrepancy between how US and EU sees civil agreements.
In Europe it's perfectly normal to be bound by terms of a paid service. I would never expect to avoid being liable for payments for services rendered only because I didn’t enter a cc number before exceeding free-tier limits.
Even in the comments below people are stating that this bill is valid only if they want to continue using the service.
I dunno. I'm in the US and I feel that those saying they wouldn't expect to be liable for services they consume beyond free tier limits sound awfully entitled.
I mean, it's a sleazy practice, especially considering how they word their emails, but I wouldn't expect something for nothing just because I hadn't set up a billing mechanism yet.
Blacksmith are wrong, but also they’re a YC company- they may be young founders that haven’t run a SaaS before and genuinely don’t know how to handle free trials.
This seems like the wrong tactic, even in a money grabbing sense.
Either say one month free and end after that, or end after the free data amount. If the customer finds value they will sign up and be less likely to look at their credit card bills.
This just pisses people off and puts the cost front and centre so they can't ignore it.
I'm old enough to remember that capitalism used to always feel like this. Even worse. Predatory subscriptions, targeting minors in some cases (remember phone ringtones?), toll numbers, unclear fees, surprise bills (my favorite - roaming fees in your home country because the phone stayed logged into the foreign network after an abroad trip). Silicon Valley/YC style startups felt like a breath of fresh air. Generous return policies, subscriptions can be canceled monthly and even do so automatically in a lot of cases (eg disabled credit card). I guess the fact that a lot of what they offered has near zero marginal cost helped. AI changed that part, so we're back to what I would consider the natural state.
The least-resistance path out of such a bad equilibrium is regulation. And they did add a lot of protections in the last decades, which probably helped too. Some would say places like the EU even added too many. But I'm pretty sure that "if you're using our product, you need to pay" would fly even in the most customer-friendly jurisdictions today.
It would, and the EU would actually be worse on this. In many if not all EU countries (DACH at the very least) you'd have to pay because you agreed to pay, and you'd have to pay all court and legal fees you caused as well - while in the US the company is more likely to just forget about it and try for another customer.
You can't start a contract merely by reading a comment, but you can start one by using a commercial service with a pricing model, after being informed that you should read the contract.
By reading this comment, you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
>Blacksmith helps us build faster, and once we agreed to pay for the (actually useful) service, their support got friendlier. So… we’ll probably switch back.
well, that was one of the most disappointing conclusions that i have read recently.
i am almost more frustrated with this decision than i am with the billing shenanigans. please do not reward this scummy behavior.
Why am I not surprised that its a YC startup? Lately, being a YC startup seems to have become a negative signal for me, far too many grifters are getting funded by YC, it seems.
Billing statements disguised as marketing nudges is a cromulent business practice until the SaaS start sending bills to collections.
I think the author is being kind, both to themselves and startup practicing dark patterns. He walks through his own thinking, raises important questions and also gives the benefit of the doubt that I wouldn’t give.
IMHO, the article gets ahead of criticism well: accepting the valid critiques while also confining the weird/lazy ones to downvotes.
1) They intended a bait-and-switch, where they were going to go after everybody for non-payment. According to the article, they might not even have a leg to stand on here.
2) If you take the quote from the company at face value, they realized that their free quota would be insufficient for conversion for some customers, and decided not to shut off services in the middle of evaluation. In this instance, the bill is a communication in advance -- if you provide a credit card in order to keep using our services, we want to get paid for everything you used after the free limit.
Now, you can argue (and many are) about whether this is a good business practice or not, but it really doesn't matter. After making the front page of hacker news, it's probably not one they're going to continue, simply because now that everybody knows about it, you'll probably have a lot of bad actors doing multiple signups, just to siphon off as much token usage as possible.
> In this instance, the bill is a communication in advance -- if you provide a credit card in order to keep using our services, we want to get paid for everything you used after the free limit.
That would be much more acceptable. If it worked out and you want to continue, you won't have a problem paying for the overage. If you decide it's not for you, then you can walk away and owe nothing. If it were communicated that way it would be a different situation.
I mean, the dude didn't even try to say he wasn't warned:
> We're writing to inform you that you've used up 80% of your free minutes for the forestwalklabs org this month. Please add a credit card on file to avoid disruptions to your service.
Then while they kept using it, if you read between the lines, they kept receiving more warnings.
> A couple weeks later we got a “You’ve spent $500.60 on Blacksmith this month” message, which didn’t seem true since we were on the free trial still. Maybe that was what it would have cost if we weren’t on the trial? Anyhow, it was one of an embarrassingly large number of usage-warning emails in our inboxes, and this one neither had a credit card nor impacted production users.
And he freely admits:
> And let’s be clear: our agents run a lot of CI jobs, so we did expect to hit the limits of the free plan. We used the service and got value for it. So it’s not inherently dishonest, just surprising. My read is that they can do this.
Again, the only question is whether the company would have tried to collect if the customer stopped using it cold turkey. We have no way of knowing whether that would have happened or not.
And, really, at this point, in one way I think it doesn't matter. After having been on the front page of hacker news, the company will almost certainly work to make things clearer.
And in another way, it matters very deeply. A knee-jerk reaction to this would be to simply stop providing services after the free trial, and to point to this blog post and thread about how obviously continuing services was the wrong thing to do. They may be forced to do this if too many bad actors try to take advantage of them with multiple sign-ups.
> I mean, the dude didn't even try to say he wasn't warned:
A warning that service will be disrupted if you don't add a credit card clearly implies that service would be cut off once they used up the free time. Continuing to get service is not a disruption. Maybe you were agreeing, not sure.
The question is, is not immediately disrupting service a courtesy, or a dark pattern?
I think it boils down to intent, and now we can't have proof of intent since the OP paid up, but the wording of the messages he received could certainly be interpreted to be that the company thought they were doing him a courtesy.
If you read the other comments, you're certainly an outlier in this belief.
My own answer is that there is nothing (incromulent? uncromulent? well, anyway, not cromulent) about suing people who knowingly and deliberately racked up debts with you, but that common business practices, including the overwhelming abundance of free services everywhere in every product category, and the ability to immediately shut off internet services when you aren't paid, lead to sort of a gestalt of expectations about how things are done.
The article itself says that the terms of service only allow billing if a payment method have been provided, so suing absent that provision would probably be a non-starter anyway.
On the bright? side, suing would definitely keep them on the front page of hacker news longer.
I find that exploiting a difference between expectations and reality is a common way to make money, but I'm no good at it myself. I'm cursed with engineer-brain instead of business-brain.
> exploiting a difference between expectations and reality is a common way to make money
Maybe? Or maybe expectations catch up with reality? I mean, does anybody really go into a car dealership thinking the dealer isn't going to do their best to rip them off? Or believe that "their" real estate agent is "on their side?"
I think most people just accept that the world is full of rip-off artists.
Which is kind of sad, really. If one person pushes back on bad billing, the company still makes money, even if it has to do a refund.
But if everybody were to always push back? It wouldn't be worth it to try to tack on extra fees for non-rendered services.
Yes, lots of people who don't frequent HN think their real estate agent is on their side, and think car dealerships just sell cars for a fair price. They might even think politicians make laws to make society better.
That's why all these groups of people are able to make so much money. If everyone expected a car dealer to rip them off, car dealers wouldn't be able to rip anyone off and there wouldn't be so many of them.
They got done in by a massive class action, that I was tangentially a beneficiary of, not because of the minutes claim, which was standard practice, but because they had failed to provide anyone with the cost of the data.
I think I paid them 300 bucks or something in the end. After further letting a 600 dollar agreement go to collections and settling with collections for 50%.
reply