Originally presented as a PAYMNENTSfn Fireside Chat, listen as Joel Taylor from TaskRabbit and Kevin Mooney from Bandcamp discuss and answer questions about legacy payments code and the impact to customer experience.
Have questions that aren't covered here?
Join us in our Slack community and get your questions answered!
Want more?
You can also watch all of our PAYMENTSfn Fireside Chats here.
Rough Transcript (edited for readability):
Peter Mollins:
Well, hello everyone, and welcome to a new PAYMENTSfn Fireside Chat series. My name is Peter Mollins, I'm with Spreedly, and before we get started, if you'd like to submit a question, please go ahead and click on the Q&A option at the bottom or your screen. We have a team here that's going to be taking a look at those questions and passing them on to the panelists for further discussion. You could also actually submit questions if you're part of the PAYMENTSfn Slack community, just go to Ask PFN, the channel Ask PFN, in that Slack workspace, and feel free to ask questions there. We'll also be monitoring in that location.
Peter Mollins:
So we'll definitely do our best to get to as many questions as possible during today's webinar, and if you're not able to stay for the entire webinar, don't worry, because we'll be sending out a viewing link that you can access after the event. So without further ado, let me introduce you to today's panelists and a little bit about the topic.
Peter Mollins:
Again, reminder about today's topic, which is a fireside chat looking at legacy payment code. Joining me today first off, Joel Taylor, who's staff software engineer at TaskRabbit. Joel's a software engineer who focuses on building robust, optimal and maintainable payment systems. For the past two years, Joel's been leading development of TaskRabbit's global marketplace payment service, and he's passionate about all things payments, he enjoys sharing knowledge and developing solutions for the unique technical challenges surrounding the payments domain, and we've seen a lot of activity from you, Joel, in the PAYMENTSfn Slack, so thanks very much for that, and welcome to today's fireside chat. [crosstalk 00:01:30]
Joel Taylor:
Great to be here.
Peter Mollins:
That's great, thanks, Joel. Also today, Kevin Mooney, who's developer/manager at Bandcamp. Kevin's an engineering manager and individual contribution on Bandcamp's payments teams, and he lives in Austin, Texas. So Kevin, welcome.
Peter Mollins:
Also with me is DeeDee Lavinder, she's senior software engineer at Spreedly. She's on the vaulting team at Spreedly, where payments methods definitely have the spotlight. The juxtaposition of analytical thinking and creative problem-solving is really where she's most happy, and she's thrilled about finding the sweet spot in technology. She's also a director with Women Who Code, and working on their mission to inspire women to excel in technology careers. So DeeDee, welcome.
DeeDee Lavindar:
Thanks, Peter.
Peter Mollins:
Excellent. So let me kick off: the topic today is around legacy code with payment systems, and the reason that I chose this topic, that we together chose the topic, was because combining two top talks that were given, one in 2020 at PAYMENTSfn and one at the 2019 edition of the PAYMENTSfn conference, so Joel, you had spoken about the idea of the haunted forest of legacy code, which was a great session, nice comparison between this hunted forest and what goes on in this mysterious woods.
Peter Mollins:
And then also, Kevin, you were talking in 2019 about some of the challenges that come with when you build fast and start early, and then as the company begins to develop and mature, what happens and what kind of strategic changes need to go in place. So with that as kind of a background for why we chose the topic, first off, maybe I can turn to you, Joel, and ask questions. TaskRabbit, it's a marketplace, but can you tell me what makes a marketplace different, from a payments perspective, versus a more traditional merchant?
Joel Taylor:
Certainly. The fundamental difference there with a basic merchant is that you're accepting payments, generally your debit or credit, and the marketplace introduces the second party now of sending payments out, so we're doing disbursements. Which is a pretty basic difference, but then has a lot of subtle differences too, so now you have different legal implications. Your fraud and law strategies are going to be different, because now the money's moving in multiple directions, you have totally separate accounting challenges, but it all just boils down to how you're responsible for taking and sending money.
Peter Mollins:
Okay, great. For those not familiar with TaskRabbit, would you mind just giving a quick summary of what TaskRabbit is?
Joel Taylor:
Yeah, we are a marketplace that provides taskers a platform to be hired to do various jobs around the house. The goal is making everyday life easier. Clients can sign on through the application, look up someone for a variety of tasks, hire, get it done, and move on.
Peter Mollins:
Great. Kevin, same thing for you, Bandcamp, it's also a marketplace, right? Though with a very different focus. Maybe if you could tell me a bit about Bandcamp's perspective on a marketplace.
Kevin Mooney:
Sure. Like TaskRabbit, Bandcamp is a marketplace, artists come to Bandcamp.com, they add digital music and physical merch, and those goods are sold on the site, so Bandcamp provides the platform for artists to sell music and merch. Similar to what Joel said, we do have unique accounting challenges and unique payments challenges that come from being in the middle of a transaction between two other parties.
Peter Mollins:
Great. And just for context, DeeDee, as well, maybe you can give a quick summary of Spreedly and your focus within Spreedly.
DeeDee Lavindar:
Sure. Spreedly is primarily an API that connects merchants to their choice of over 120 payment gateways across 100 different currencies. We also can safely store and tokenize customer payment data to help reduce merchants' PCI scope, and the team I'm on focuses on primarily on payment methods of all kinds and the vaulting of those payment methods.
Peter Mollins:
Got it, okay, terrific. Joel, when you're talking about the two-sided marketplace, you specifically cited the idea of disbursements, so payouts. So how do disbursements work when you're working with those taskers?
Joel Taylor:
The way it works for us is that, at the end of the task, there's an invoice that happens, and then we charge the clients, and then that goes into a bank transfer. It varies based on the country that you're in, the underlying protocol of it. We do a rolling payout schedule, so it's going to vary by country. In the marketplace vernacular, they call this a destination charge, which is a syntactic sugar for saying that you do a charge in transfer, and those funds are generally held onto or reserved so they don't get spent elsewhere. It's a pretty straightforward payment transfer process.
Peter Mollins:
Okay, great. Kevin, you've got a model where you're paying out to musicians when they purchase the music and the merchandise. Similar approach?
Kevin Mooney:
Similar, but not the same. We actually do all of our payouts through PayPal. Well, not all, but most of our payouts through PayPal, and so what we ask for when an artist signs up is for their PayPal email address, and we store that and then we can pay them out periodically. Typically, an artist gets paid within a couple days of them making the sales.
Peter Mollins:
Okay. Just thinking about the experience, when we think about merchants, we often talk about the payments experience and how that delivers a good customer experience, but in a marketplace, you've got two sides of that customer experience. Do you think about it, and do you think about the disbursements? I guess this is both for Joel and Kevin. Do you think about those disbursements in that sense of how you can deliver a great experience?
Kevin Mooney:
Yeah, absolutely. One of the main concerns for us has been making sure that artists get paid as quickly as possible and so when we started out, we handled payouts just by allowing payments to be made via PayPal directly from the buyer to the seller. That would've been just a direct payment, so we were not in the middle of that transaction at all. It was almost like if somebody buys you a slice of pizza and you pay them back with Venmo, it was a transaction like that. The pizza guy is not in the middle of that.
Kevin Mooney:
So artists really like that, because they got paid immediately; the moment that music sold on the site, they had enough money to buy a six-pack or pay their rent or whatever they needed to do with that money. When we did that migration, going from sort of a direct payment method like that to one where we are the merchant of record, one of the main concerns was absolutely making sure that the artists' experience didn't change too much, that they were still paid as quickly as we could do it.
Peter Mollins:
Okay, great. And Joel, any additional thoughts on that?
Joel Taylor:
Yeah, actually taking a page from Kevin's talk, he mentioned the importance of empathy within this process. I think there's a lot of empathy that has to be had around the way fees are taken out too. So if you don't have a transparent way when you're doing your disbursements of making sure that your ... and you just understand all the service fees or your trust fees that are happening there, it really leads to some disgruntled people ... chances are, in a marketplace, the end users could be making their livelihood off of this. So getting that flow that's transparent and has the visibility so they can see where the money is at any given time is really important, so supporting that.
Joel Taylor:
And then there's also tax considerations; the way you set that up is really going to impact the way you do taxes, which is always good to happen. They'll come and get you at the end of the year, no matter what.
Peter Mollins:
Yeah, yeah. Okay, great. Kevin, last year in your session, you spoke about the challenges of disbursements, and one of the things that was really interesting for me was how things changed over time. Initially, one of the implementations that you had for disbursements was this concept that was internally called Yoinks. Do you mind just kind of recapping what that was, and how it matured from that idea?
Kevin Mooney:
In the very early days, I kind of describe the first half of that where an artist, a buyer and a seller would make a direct payment, but you might understand that Bandcamp did not get paid in that case. And so the way that Bandcamp would get paid in those early days was by maintaining an account for each seller. We'd keep track of how much they sold and the amount that they owed us, and that would be our rev share rate, which is 10 to 15% of revenue.
Kevin Mooney:
And so if the current transaction was less than the amount that they owed us, we'd consider sending that payment to our own PayPal address instead of the artist's because the trade-off for that is that it can be a little bit confusing. We had messaging around that, but I think that, for the most part, artists actually still really like that, because the vast majority of the time, they got paid immediately.
Kevin Mooney:
In order to support additional payment methods and add new product features, and also to make sure that we were capturing our rev share in a way that was fair to us as well, we did end up making changes to the way that our payouts worked. So instead of doing direct payments, we became the central, the hub of that transaction, where artists would be paid by us and we would be paid by the buyer. And so yeah, making that change meant that we had to add a lot of software components and do a fair amount of development to get to the point where we were making the switch from the old system to the new.
Peter Mollins:
Great. Well, let me switch gears and switch over to the topic at hand, around legacy code. Appreciate all the context for a platform approach. Joel, in your session, it was based on a metaphor from John Millikin's book The Haunted Forest. Maybe you can tell me or recap, what is that parallel between the haunted forest and a legacy application, or more specifically, legacy payments application?
Joel Taylor:
I have to say, if anyone has a chance to read that post by John Millikin, consider it literature for tech writing, it's a really great piece. So the whole premise of this is that, we use this term tech debts. Whenever we're making subpar decisions or just things that aren't up to our likings, but more often than not, that tech has really drastic consequences or it has really big impact over a period of time. With payments, I think that impact is even, it's a little bit louder, scaled up, because now you're dealing with money. When things go wrong there, when there's mysterious things moving around in your system, it tends to be pretty impactful and significant when things go wrong.
Joel Taylor:
So it's not just tech debt anymore. And when there's entropy, it's a compounding thing that happens, so it's going to continue to grow, and if it's left unmaintained, it can grow into this beast and then you have this haunted forest, per se, where you have engineers who are afraid to touch it, it's resistant to change, it adversely affects your image and your morale. It's typically prone to being avoided because it seems to be a scarier part of the code. I think that metaphor really fits into that nicely.
Peter Mollins:
Great.
Kevin Mooney:
Can I interject or add to what Joel said? Because legacy code is one of my favorite topics.
Peter Mollins:
Yeah.
Kevin Mooney:
I think Joel mentioned the concept of technical debt being something that you accrue by making decisions that are sub-optimal early on. But I kind of want to ride to the defense of technical debt a little bit because it's a tool, right? It's almost like financial leverage, where you are making more work for yourself later by taking a shortcut, but sometimes you have to taken that shortcut for you to survive to a point where you can have the resources to do it differently later on.
Kevin Mooney:
I think that there's not a right or wrong way to ... there's not a universal right or wrong way to talk about technical debt. It's not always bad, it's not usually good, but if your organization is small and you don't have a lot of resources, doing things quickly and without thinking about maintenance performance scalability, to an engineer, that seems scary, but that sometimes is the right call because it's just necessary. Yeah.
Peter Mollins:
Yeah, that's great. DeeDee, you mentor to a lot of developers that are starting up, does that resonate with you, Kevin's defense of legacy code?
DeeDee Lavindar:
Yes, actually, it does. While I agree with everything that Joel and Kevin have both said, I'll also add that sometimes technical debt comes because of a change in circumstance, something that was chosen optimally, that was a good choice at the time. As circumstances change down the road, can become problematic, but not because it was a poor choice at the time or because you were moving too fast or being sloppy. So it's sort of like a natural state of expanding your code and asking your code base to do more and more things, adding features, et cetera. So technical debt shouldn't necessarily be the bad guy.
Peter Mollins:
That's great. Well, terrific. We've touched on a couple of the reasons for technical debt, good or bad, emerging. I think it's absolutely right to talk about sometimes it's because it's quickly written or sometimes it may be because it's not maintained. Or like you all have been saying, or business circumstances change. Any other context on some of those causes? I'll just throw this out to any of you three for other causes for where technical debt comes from.
Kevin Mooney:
I think that sometimes it's a weird metaphor, right? Because it's almost right, but it's not completely right. I think a lot of people talk about technical debt in terms of having a code base that they no longer understand. And I think that part of that comes from the fact that ... something I've observed is that reading code is a skill that's different from writing code. You can hire engineers who are really good at writing code, and it's not a matter of being good or bad, but it's just a matter of skill development, who haven't developed the skill at reading it.
Kevin Mooney:
And I think that, for a lot of reasons, it's just easier to engage when you're writing something and testing it and thinking deeply about it, than when you're sort of reading over something that somebody else did and trying to run it in your head, instead of running it on a computer and watching it evolve in real time. I think that part of it is developing a skill set for yourself as an engineer, if you are one, or for your team of engineers, developing a skill set that includes reading code is really valuable for mitigating the complexity of big, old systems, because then you won't even notice that it's technical debt. You'll just have people who read the code and understand it.
Kevin Mooney:
That's not a super engineering thing, that's just another skill that can be in the toolbox.
Peter Mollins:
Yeah, that's quite interesting, because when I think about reading the code, you can think of it in two angles, one is reading it technically, understanding the technical structure and function of it, and then understanding it from the business perspective of what is it trying to achieve? How do you connect in, this is for all three of you, or for any of you three, how do you connect them to business users to make sure that reading skill of the code can be even enhanced further by understanding rhe business context of the application? Do you partner with business analysts or is it a matter of documentation, or is it developers just building that muscle of knowing that this is a business, it's being written for a business reason?
Joel Taylor:
I certainly think there's value in understanding the business of aspect of whatever you're implementing a feature, especially with payments. Understanding the underlying systems of what you're doing, understanding what that charge actually is and what can go wrong, understanding what can go wrong in the disbursement sense. All the domain language, it's going to matter as you grow and things grow in complexity, is going to matter to your stakeholders and your business as well, especially when you start having more advanced accounting techniques having to happen. Understanding those domains and being able to map them and think about them really helps with the whole process.
Peter Mollins:
Great. Go ahead, DeeDee.
DeeDee Lavindar:
I was just going to say that technical debt can sometimes represent code that has become brittle or hard to extend, and so if you're going to refactor legacy code so that it's more extendable and more scalable, that deep understanding that Kevin's alluding to, that comes from being able to read code well, your functionality, essentially the end user, their experience shouldn't necessarily change. It should just mean that your code is more stable and able to be added to, that you can add more features now, you can extend it more easily.
DeeDee Lavindar:
For the end user, the technical debt should not necessarily be a factor. The complex code is still, it's providing what they need, it's just that for adding onto the code base or adding new features, making it more stable and more extendable is the goal.
Kevin Mooney:
That's really insightful. I think also to ... it makes me realize, also, that there are more ... my favorite thing is to make a change where nobody knows the difference, where it's impossible to tell that change has been made. That means that it worked. It's really interesting, because organizationally, your engineering team and your product team, in general, your business teams are going to have different ideas about the value of making a change that's not visible, because there's a sort of, it's an iceberg, where the tip of the iceberg is the client functionality, and then underneath it, the big part of it that's under the water, is a lot of engineering that needs to sometimes be updated to add new features.
Kevin Mooney:
What DeeDee's saying about adding, making that stuff more robust, more extensable, means that we can build better features and more features more quickly. But there are these weird halting moments when your engineering team is doing a lot of work in producing things that they're very proud of, but nobody can tell the difference.
Peter Mollins:
That's great, that's a great point. We actually have a question from the audience. Is technical debt actually a solvable problem, or is this just ... as companies grow, there will always be that bottom of the iceberg, I guess, or the aspect of the bottom of the iceberg that's just forever, it's just there. Maybe Joel, do you want to chime in on that one?
Joel Taylor:
I really agree with what Kevin brought up, that it can be a tool in leverage. I don't think it's something that you want to fully eradicate, especially when you're moving quickly. There's definitely trade-offs that can happen, need to happen. I don't know if it needs to be solved, I think it can be wrangled or it can be made as a useful tool. I think it does happen naturally, you can't know what you don't know about your features, about what's going to happen, where you're going to grow. I like to focus more on this topic of extendability, is making sure your systems are tested, that you have that ability to change and pivot without having to make guesses or hope that you don't break things. I think with great test coverage and great strategies like that, you can make your life a little bit easier.
Peter Mollins:
DeeDee, you're coming from Spreedly, so Spreedly, like many other approaches, is an API-based platform. One of the advantages in tools like that is just the ability to obfuscate some of the complexity below the surface. Do you see that a lot, of customers coming to you looking to focus on business challenges and business problems and trying to ... and perhaps hide some of the complexity of connecting into legacy systems?
DeeDee Lavindar:
Absolutely, especially when you offer, like, one integration will solve all your payment processor problems. Because we offer a single integration and we connect to all these payment processors on the back side, folks often use Spreedly for more than one payment process or for many payment processors. They want their experience to be smooth and to be consistent, so these are definitely considerations on our end.
Peter Mollins:
Great. So let me just switch gears again. One of the things I was really interested in is about distributed teams and thinking about how much that could contribute to technical debt or complexity within code. Joel and Kevin, you both have teams that are very widely distributed. What are some tactics that your teams do in order to make sure communication is there, to avoid unnecessarily complicating applications or leading to more technical debt that perhaps would be ideal? Feel free ... Kevin, go ahead.
Kevin Mooney:
Okay. I think ... Bandcamp has been distributed since the beginning of the company, and during that time, we've developed a few tricks that are in our toolbox that make coordination sort of a little bit easier. I think maybe the one I like the best is that we've established a single time zone for the whole company, which happens to be U.S. Pacific time, so daylight savings is still kind of, it can be tricky, but for everybody, we just sort of communicate our times, our meetings, any time that we need to be synchronous, we always communicate that in U.S. Pacific.
Kevin Mooney:
I'm in Texas, so it took me a few days to get used to it, but it's much, much easier than trying to figure out what time zone this other person is in and what time zone I'm in, and then what that time really is, so that we meet up. So prevents a lot of problems with miscommunication or not being able to meet up.
Kevin Mooney:
The other thing that we try to do is, on a team, we try to make sure that people just do have at least a few hours of overlap with everybody else on their team. So if something does come up where we need to have an immediate conversation or a real-time conversation, we can do that. If everybody's in the same time zone, I know lots of people are working from home right now, so if everybody's in the same time zone, it's kind of easier and it's not really a consideration. But I think that making sure that everybody can commit to just having some overlap helps a lot.
Kevin Mooney:
The other thing is that since we are fully distributed and have been since the beginning, a lot of rituals around hiring and interviewing and things that are really office-based are things that we've already established ways to deal with. So that's helped, too. Not sure how that helps with technical debt. We do use Slack, Zoom, and Fabricator, which is a ticket management system, and using those three tools really mitigates any downside of having a remote engineering team.
Peter Mollins:
Great. And Joel, you're working remote as well.
Joel Taylor:
Yeah, yeah. We haven't been remote as long, [inaudible 00:29:40] about two or three years now. But echoing a lot of what Kevin's saying, a lot of tools that we're using make it easy, but I think the benefit with remote work and disputed teams is that you can make use of [inaudible 00:29:53] communication a lot more, so we're seeing folks to write down their ideas or to write down what they're trying to do is a lot more challenging that just kind of hopping into a meeting and talking about it. We have some process around that that helps, as far as with code and whatnot.
Joel Taylor:
But even a struggle we've had with our payment system is making a pretty large change, and there's a certain way to use the API, and it got misused in our code base because our engineering org's pretty large and some people didn't get the message. So then I added some automated tooling to warn when there's a misuse of our tools. So things like that really help when you can send out memos and emails, although adding things in your deployment process or your testing process that can catch these things early on, I think, are really helpful and reduce bikeshedding or anything that has to be a needless conversation that could just be done by automation.
Peter Mollins:
Great. And DeeDee, Spreedly has a hybrid approach, right? Some people are 100% remote, and these pre-COVID people were in the office 40% of the time for those that work near the headquarters. Any similar thoughts about working with distributed teams?
DeeDee Lavindar:
Because we have a hybrid model, we have other challenges. One thing is that when people are in person, it's hard to remember not to talk about critical things that the whole team has to be a part of if some of you are remote and some of you are local. So we do try to keep that remote mindset even when we're at the office. It's just an added challenge to having the hybrid model. For the most part, though, Spreedly is pretty well set up with all the remote tooling, et cetera, because the local folks are only in the office two days a week. We do have that kind of remote-first idea.
DeeDee Lavindar:
Although the quarterly get-together is when everyone comes in from out of town, the fully remote people, and we have that in-person time. There is a different quality to that time that you have together than when you're just on Slack and Zoom day to day. So we try to save brainstorming sessions or things that are somehow easier to do in person, we'll have a pretty full slate of in-person meetings to hash things out.
Peter Mollins:
Great. Well, let me go back to some of the impacts from legacy code or technical debt. Joel, in your presentation back in May, you have an example around disbursements, was the example you had, and so you wanted to pay out to a tasker, but found that sometimes there were significant delays in actually getting the money into the taskers' hands. Can you tell me about how legacy code impacted that, and what you did to address that?
Joel Taylor:
What's interesting about this one is that, with disbursements sent to Temporal, it is already difficult to test. We tested the logic around that where it's going to display properly, but there's nothing to check what happens when it's seven days after a disbursement should've been settled and it's not settled yet.
Joel Taylor:
The legacy impact on this was more about a misunderstanding of how the system should work. At that time we had two processors, and there was a little bit of deviation in the way those two behaved and there was some drift in the code, but there was no documentation, there wasn't anything explicit about why that drift was there. We introduced engineers having [inaudible 00:33:40], and then now we have this code that's mysterious and it stayed pretty, you don't want to touch it because it just works. We don't know why that drift is there.
Joel Taylor:
It's all those compounding factors made this one area of code, no one just really knew what was going on and why disbursements behaved that certain way. So it's kind of a really silly bug when we found it, and we were able to fix it. But it's most just around the cultural misunderstanding of our payment system.
Peter Mollins:
Got it. And speaking of differences, payment processors themselves, they're very different, as you just cited. They have different APIs and they all have different behaviors. Was that the same example? I know you brought up previously in your presentation about the differences between processors impacting the system, was that the same example, or was that a different case?
Joel Taylor:
Both very related, they kind of both drilled down into that. In that particular case, that was our weblink notifications behaving differently. One processor just told us that he had unexpected behavior that we didn't anticipate. That was very surprising.
Peter Mollins:
Got it. Kevin, going back to your 2019 session, there was a challenge that you discussed around payments applications supporting a single currency for stored payments methods. Can you tell me a bit about what happened there and how you approached that?
Kevin Mooney:
Sure, yeah. The challenge there was that we were in the process of adding, actually, a new feature, because everything is features for me. We were adding the ability to accept credit cards for certain types of goods. Prior to that, we'd only accepted PayPal. And so I mentioned at the beginning that we do payouts with PayPal, so it's nice to have everything going to a PayPal account. We were using their ... at the time, it was called Payflow, I believe it's called PayPal Payments Pro now. We were using their credit card service.
Kevin Mooney:
Not to get too far in the weeds with it, but the way to store, to vault a card, quote-unquote, in that service is to store a reference value that they return after you do a transaction. The thing that we discovered was that using this reference value for subsequent transactions only works if the future transactions are performed in the same currency as the original. That's totally fine for most online merchants because most online merchants aren't going to change their currency very often, if ever, but in our case, a single check-out could have multiple currencies, not to mention subsequent transactions. That's because we're a marketplace that has lots of sellers on it and a single aggregate check-out.
Kevin Mooney:
So the thing that brought us to Spreedly was our requirement that we vault cards in a way that would let us to continue to transact with stored cards in multiple currencies. We were able to dot hat with Spreedly's payment method distribution product, so with the PMD payflow receiver, we're able to vault payment methods in Spreedly and then use that receiver to make pretty much arbitrary payflow transactions in the future using the tokens that we get back.
Kevin Mooney:
I don't recall the details, but I believe that there wasn't a payflow receiver when we started, so we had to work with the customer success team at Spreedly to get that receiver built. It was built, and we've been using it for years now. It works.
Peter Mollins:
That's really interesting. Thanks, Kevin. Some of the things that jumped out from that example, I remember in your session, and Joel, also in your talk, was just how changes in adding functionality or making changes to existing functionality within the payments application can have an impact outside of the payments organization itself. I think there was one example where support tickets had dropped by 40% as a result of disbursements going through more effectively. I got to imagine, for both of you, when you have an impact like that outside the organization, that you can get a lot of friends in finance and customer success and other places.
Peter Mollins:
Maybe Joel or Kevin, any examples that you might highlight of where that impacted, where changes to a payments application had a positive impact outside of the organization?
Joel Taylor:
Yeah, two cases that come to mind immediately are the example of the customer support tickets. That was a one-line change where it was just some copy that was wrong, a conditional, because if you tell your users to contact support, they're going to contact support even if things aren't wrong. So that was a pretty simple one where it was a misunderstanding over a payment system.
Joel Taylor:
And then another fun one is, recently discovered that some charges are endlessly charged. This is a different case in somewhere else, but there's no limit to how many times the charges were happening, so if you think about the number of years of some charges being retried over and over again, you're going to hit processing fees. In this scenario, I discovered that there was about $80 in fees per day happening due to failed charges, to the point where a lot of these charges had even exceeded a potential revenue. Once we realized that, we put a cap on how many times we're going to attempt a failed charge. [inaudible 00:39:42] savings, so that just impacted our accountants, made a lot of people happy, but it's kind of silly how something like that can end up just eating so much money, but also be hidden in the weeds.
Peter Mollins:
Great. Kevin, you discussed the impact of payment [inaudible 00:40:02] on product development teams. Anything else to add around that?
Kevin Mooney:
Sure, yeah. One example that seems kind of relevant today was the way that we were able to respond to the global pandemic that's happening right now. I know everybody wants to talk about this, but I think what we did was pretty meaningful. Since this has started, touring musicians and tourist artists have been really badly impacted because they can't play anywhere, there are no venues. And so what Bandcamp has been doing is 0% rev share days for, so far, for the last few months since the end of March. We've done one in March, May, June, July, and we did one last Friday at the start of August.
Kevin Mooney:
It turns out that adding the ability to globally turn off rev share is a feature, and it's one that requires specification, development, testing, and given how quickly everything sort of developed this spring, it was something that we wanted to do quickly. It's exactly the kind of thing that having a big, scary, old code base and a lack of expertise would inhibit us being able to do with confidence.
Kevin Mooney:
But we have managed to, we did it, and so we were able to add a feature, test it, and make sure that we were able to turn off our rev share, turn it back on, our payouts for sales that happen during that period are 100% of the shares. And it's been really effective, and I think that that really speaks to the importance of being able to make an investment in having a robust and extensable code base. If that rest of the iceberg wasn't right, we wouldn't have been able to turn around a feature like that in that amount of time.
Peter Mollins:
So true. You need to focus on what the business aim is there, and supporting your artist is absolutely critical, so that's great. If you don't mind, maybe I'll just conclude here with a big question, if you don't mind. Open for all three, rewriting or refactoring, how do you decide when it's the right to rewrite or refactor an application? Do you have thoughts on that? DeeDee?
DeeDee Lavindar:
Definitely a time to refactor is when you can't make a change that you need to because you're limited by the code that's there. But that's not talking about your entire code base, we can often refactor small portions of it for huge gain. Sometimes when you're in a code base that feels frustrating, you just want to start over, but finding those areas where you can put in, where your return on investment is going to be high, it's probably a good place to start as far as effort and payout.
Peter Mollins:
Right. Kevin or Joel? Joel?
Joel Taylor:
I agree with DeeDee on that. I think refactoring, where possible, I tend to favor that. But I do think there's a point time where, maybe, a payment system has drifted to the point where it's not usable or can't serve your needs to the point where it's just inhibiting so much. Refactoring could make it more pleasant to work with, but it's not going to get you to the desired outcome. In those cases, I think it's a plausible solution.
Kevin Mooney:
I concur with both of you. I think that, in some ways, effective refactoring is sort of a rewrite in slow motion, where at the end of a long period of time, it might be years, you really do have a completely new system, something that's completely different than what you started with. And that also, refactoring in cases like that, when you have the opportunity to do so and the perfect confluence of resources to do it and no backlog, or not much of one, and all of those other things, I think that that's really the best approach to getting to something that's robust.
Peter Mollins:
Terrific. Well, we're at the end of our time, so I want to send out a big, big thank you to Joel, Kevin, DeeDee, and to the entire audience, thank you very much for participating and for your questions and for joining us. As you know, this is going to be an ongoing series, this is the second in the series, with the next episode airing in September. So be sure to check out the PAYMENTSfn Slack community to get details on September and October and all the upcoming events.
Peter Mollins:
If you enjoyed the webinar, definitely encourage your colleagues, your payments colleagues, to join the community, that gets them free access to all of the content. It's free, so they can join events like this as well the annual conference that takes place generally in the springtime.
Peter Mollins:
Again, thank you so much to the panelists and to the attendees, hope to see you next month for episode three of the Fireside Chat series, and also, if you'd like to speak or be a panelist for the Fireside Chat, definitely feel free to ping the team within the PAYMENTSfn Slack, we'd love to hear more about what your thoughts are and see how we can continue to share information across the community. So again, thanks, everyone.