Image source: The Motley Fool.
Origin Bancorp, Inc. (OBNK -0.14%)
Q3Â 2020 Earnings Call
Oct 29, 2020, 9:00 a.m. ET
Contents:
- Prepared Remarks
- Questions and Answers
- Call Participants
Prepared Remarks:
Operator
Good afternoon. Welcome to the Origin Bancorp, Inc. Third Quarter 2020 Earnings Conference Call. [Operator Instructions] [Operator Instructions].
I would now like to turn the conference over to Chris Reigelman, Head of Investor Relations. Please go ahead.
Chris Reigelman -- Investor Relations
Good morning, and thank you for being with us. We issued our earnings press release yesterday afternoon, a copy of which is available on our website, along with the slide presentation that we will refer to during this presentation. Please refer to slide two of our slide presentation, which includes our safe harbor statements regarding forward-looking statements and the use of non-GAAP financial measures. For those joining by phone, please note the slide presentation is available on our website at www.origin.bank. Please also note that our safe harbor statements are available on page five of our earnings release filed yesterday with the SEC.
All comments made during today's call are subject to the safe harbor statements in our slide presentation and earnings release. I'm joined this morning by Origin Bancorp's Chairman, President and CEO, Drake Mills; our Chief Financial Officer, Steve Brolly; President of Origin Bank, Lance Hall; our Chief Risk Officer, Jim Crotwell; and our Chief Credit and Banking Officer, Preston Moore. After the presentation, we'll be happy to address any questions you may have.
At this time, I'll turn the call over to you, Drake.
Drake Mills -- Chairman, President and Chief Executive Officer
Thanks, Chris, and good morning. Our third quarter results continue to solidify confidence throughout our organization that we have the right people, the right relationships in the right markets, supported by great infrastructure, strong capital and credit quality to not only work through this pandemic, which strengthened relationships while remaining in a position to capitalize on opportunities. This is detailed through our unique and defined culture shown on slide four. We'll get into more detail later in the presentation, but I want to provide a high-level overview of our strong results for the quarter, starting on slide five. Total assets increased to $7.1 billion, while net income came in at over $13 million or $0.56 of diluted EPS. Our pre-tax pre-provision earnings increased finished in the quarter at $29.9 million and our net interest income, also a high for our company, is up by approximately $4.3 million from the prior quarter to $50.6 million for the quarter.
Even through the uncertainty of 2020, there are many bright spots throughout our markets in the company. Our bankers remain extremely focused on strengthening relationships, and our mortgage and mortgage warehouse teams have done an incredible job of taking advantage of the current rate environment and capitalizing on the disruption in the marketplace. As we have stated since the beginning of the pandemic, we remain committed to the health and safety of our employees and customers to being a partner within our communities, to protecting our balance sheet and taking steps to manage expenses. I'm proud of the results of the quarter and optimistic about the many opportunities before us.
I'll turn it over to Lance to go through how we support our customers as well as discussion on loans and deposits. Lance?
Lance Hall -- President and Chief Executive Officer
Thanks, Drake. I'll start on slide seven. We think we've smartly managed our clients' financial and psychological needs through the use of forbearances during this pandemic. As we've communicated over the past two quarters, our bankers were extremely proactive in reaching out to our customers when the pandemic first hit. For the second quarter, we first reported our COVID forbearances at $1 billion, or approximately 21% of loans held for investment less PPP. 60 days later, loans under forbearance were reduced almost 60% to 8% of loans held for investment, excluding PPP. As you can see, we've reduced those forbearances even more to less than 6% of loans held for investments, again, backing out PPP loans. Looking forward in -- forward into quarter four, we believe those forbearances could decrease to as little as 2.5% of loans held for investment. I'm extremely proud of how our bankers have responded over the first nine months of the year. Relationships are at the foundation of who we are as a company.
And our teams throughout all of our markets have been hyper-focused on consistent dialogue and delivery with our clients. slide eight highlights our deposit trends, deposit cost reductions and continued opportunity to further drive down deposit costs as time deposits mature. While growing deposits $1.17 billion year-to-date, including $400 million in Q3, we have lowered our cost of total deposits from 104 basis points at the beginning of the year to 42 basis points at the end of the third quarter. Our average rate on time deposit renewals over the past 30 days has been 26 basis points, which indicates continued opportunity to reduce costs as the weighted average yield of our time deposits maturing in Q4 is 1.57%. Effectively managing our funding costs will continue to be a huge priority through the rest of 2020 and into next year. We also continue to have success in driving noninterest-bearing deposits as we have grown NIBs approximately $484 million year-to-date, bringing our NIB mix to 30% of the portfolio.
Moving to slide nine, shows where we have seen loan growth by category. Mortgage warehouse continues to be a significant win for us. We ended the quarter with over $1 billion in warehouse outstanding balances as we have effectively delivered on both needs derived from market rates and new client acquisitions, driven by dislocation in the market. This slide also shows we've been able to grow our loan portfolio 4.5% year-to-date outside of warehouse and PPP. From a strategic planning perspective, we remain focused on making smart investments in technology that should be -- should drive a heightened customer experience and create efficiency in the company through partnerships, focused on data management. The expansion of commercial and consumer lending platforms and the integration of robotics to drive process automation and improvement. Along with better use of technology, we're also working diligently to manage expenses in areas such as leased real estate, staffing models as well as vendor contracts. Our focus on relationship growth, technology, expenses and margin management should continue to serve us well to position ourselves for the future.
I'll now turn it over to Jim Crotwell to discuss our credit culture and loan portfolio.
Jim Crotwell -- Chief Risk Officer
Thanks, Lance. As we have previously reported, we continue to closely monitor those industry sectors that could experience a more protracted recovery from the current economic downturn, specifically the sectors of hotels, energy, nonessential retail, restaurants and assisted living. As you can see on slide 10, these sectors totaled $551 million, which is approximately 11% of total loans held for investment. Since the beginning of the pandemic, we have conducted focus reviews of these particular sectors to monitor the impact of the economic downturn and to ensure the appropriateness of assigned risk ratings. I'd like to point out that we have recently completed our annual external loan review, which validated our initial internal ratings. As Lance mentioned, we are pleased with the reductions in COVID-related modifications across our portfolio during the last quarter and are pleased with the resiliency of our portfolio, which we believe is driven by its diversity. On slide 11, we provide information on our hotel sector, which totaled $64 million or 1.3% of total loans held for investment net of PPP.
We continue to feel optimistic about this sector given the low average pre COVID loan-to-value of 41% and the liquidity in excess of $200 million from the guarantors of these loans. The energy sector, as outlined on slide 12, continues to perform well with outstandings reducing to $55.5 million, which is 1.1% of total loans held for investment net of PPP. As we have indicated in the past, we have no direct exploration and production exposure. Our nonperforming loans in this segment continue to be comprised of one relationship totaling 2.2 million, which has been a long-term workout. On slide 13, we have provided information on our nonessential retail sector, which totaled $151 million, representing 3% of our portfolio. Overall, this segment increased $4.6 million during the quarter, driven by a $6 million opportunity in our Houston market, which included strong anchor tenants, strong debt service coverage, a low loan-to-value ratio and significant guarantor support.
Overall, pre-pandemic loan-to-value and debt service coverage ratios are strong at 56% and 1.47 times, respectively. Lastly, and as reflected in the previous quarter, nonperforming loans in past dues in this sector are primarily driven by a single credit that was placed on nonaccrual during the first quarter of this year. Our restaurant portfolio is reflected on slide 14, totaled $136 million or 2.7% of total loans. I've been especially pleased with the resiliency and performance of this sector as evidenced by no past dues and no nonperforming loans. Our assisted living portfolio, as reflected on slide 15, totaled $145 million as of quarter end, representing 2.9% of total loans net of PPP. While this portfolio segment increased $4.5 million for the quarter due to funding on projects under construction, we continue to focus on reducing our exposure in this segment. Subsequent to quarter end, we were successful in exiting one of our assisted living relationships, which will result in a reduction of $9.2 million in this segment and in classified assets. While we will have a moderate writedown during Q4 related to this credit, the amount of the writedown was fully reserved for as of 9/30.
Slide 16 provides a recap of our asset quality trends. As you can see, past due loans 30 days or more continue to be positive, coming in at 0.58% net of PPP loans and are primarily driven by our past due nonaccruals. As to nonperforming loans, nonperforming loans ended the quarter at 0.6% of total loans net of PPP, down from 0.63% as of June 30 and down from 0.74% from the beginning of the year. As anticipated, net charge-off for the quarter reduced to an annualized rate of 0.15%, which resulted in year-to-date annualized net charge-offs of 0.28%. Total levels of criticized and classified loans remained stable during the quarter. Lastly, we did increase our allowance for credit losses to 1.45% from 1.33% of loans held for investment in the prior quarter, which results in a reserve of 2% of loans held for investment net of PPP and mortgage warehouse loans. The increase in reserve was primarily due to the estimated impact of COVID-19 on the company's loan portfolio combined with an extension of the reversion period from 18 to 24 months within the CECL economic forecast.
This assumption contemplates that we will return to historical loss rates in the latter part of 2023 with reversion of this level beginning in the latter part of 2021. Our current forecast status suggests the decline in economic conditions may not be as severe as originally predicted, the duration of the recovery remains uncertain. But while we do think it is prudent to build our allowance, we continue to be pleased with the performance of our portfolio.
I'll turn it over to Steve now.
Stephen Brolly -- Chief Financial Officer
Thanks, Jim. Starting on slide 17, you can see our trends related to net interest income and margin. Our net interest income was $50.6 million for the quarter, an increase of 9.3% from Q2. The main driver of this increase was an increase in mortgage warehouse lines of credit, along with deposit cost savings. In the top right waterfall, you can see the warehouse loans contributed nearly $2.7 million on an increase in net interest income quarter-over-quarter. Our continued focus on lowering deposit costs, as Lance mentioned earlier, was the next largest contributor to increasing net interest income on a linked-quarter basis, adding just over $900,000 to net interest income. From a margin perspective, we did a better job this quarter when efficient deployment of cash, which increased our margin by six basis points. Our margin in Q3 ended at 3.18%, up nine basis points from Q2. At the end of quarter 1, we mentioned the increase in Federal Home Loan Bank advances and other nondeposit funding due to the uncertainty around the economic shutdown.
NIM expansion was driven by our lower cash balances redeployed into higher earning asset classes, deposit cost reductions and mortgage warehouse lines of credit volume increases. We received a five basis point lift due to deposit mix and rate reductions quarter-over-quarter. The last thing to point out on the NIM trends is excluding PPT, we had a 13 basis point increase in NIM compared to the prior quarter. On slide 18, we showed trends of yields, cost and a profile of loans held for investment. You can see the impact that the falling rates had on the loan yields in Q2 and how in Q3 the loan portfolio did not see significant continued yield reductions with loan yield reducing only seven basis points or two basis points excluding PPP. With our continued focus on lowering deposit costs, we saw a 22% decline in the cost of deposits from the prior quarter, coming in at 42 basis points. The bottom right graph shows our mix of fixed and floating rate loans at quarter end, which has not significantly changed from prior quarter. On slide 19, I want to touch on a couple of points about noninterest income.
Pre-pandemic, we typically had about 20% of our net revenue from noninterest income, while the past two quarters, we have performed really well in mortgage, which drove the noninterest income percentage to greater than 25% this quarter. Total noninterest income was over $18 million, down approximately $1 million from Q2 due to a reduction of mortgage originations and sales from a historically high second quarter. Other significant changes in noninterest income category included a decline in swap fee income due to fewer deals, an increase in $1.2 million in other miscellaneous income this quarter. Other miscellaneous income includes income from security sales, which increased $301,000 and a $661,000 decrease in losses on other asset sales. Slide 20 covers our noninterest expenses as we look at the trend over the past five quarters. We can see progress in our operating leverage, especially in the most current quarter. While we see improvement in these metrics, our expenses did increase in the current quarter based on a few factors. Advertising and marketing increased $671,000, driven by $550,000 in contribution within our communities.
As you'll recall from our second quarter earnings call, we launched a strategic initiative with a portion of our proceeds from PPP fees, which strengthened our investment in the communities that we serve. Another expense that grew this quarter was regulatory assessment expense, which was up $544,000 in Q3 due primarily to asset growth in the first two quarters. These two items, coupled with a $475,000 in litigation settlement charges in the quarter, account for $1.7 million of the nearly $2 million increase in other noninterest expense seen in the chart on the left. We believe on a go-forward basis, we will be able to hold our expenses between $37 million and $38 million per quarter on a normalized basis.
I'll now turn it back over to Drake.
Drake Mills -- Chairman, President and Chief Executive Officer
Thank you, Steve. As we look at our capital ratios, we remained well capitalized at the end of the quarter, and we took action early this month to further strengthen our capital position. We successfully completed an issuance of $80 million in home and company subordinated notes this month. The timing was right. The market was attractive, and this was a great opportunity for us to bolster our total capital levels as well as support future growth opportunities. You can see on slide 21, the impact that the issuance has the total risk-based capital at the holding company and the bank, assuming a portion of the proceeds are downstream to the bank. In our 8-K released yesterday, we announced the upcoming retirement of Executive Risk Officer, Cary Davis. Cary has been an awesome partner of mine for over 20 years and been instrumental in the success of building our institution through multiples of credit cycles.
Cary has also been critical in building a risk and credit function that can continue to for own for years to come with the same diligence and excellence he has shown for the past two decades at Origin. Now due to our growth and increasing complexity of the environment in which we operate, earlier this year, we separated the roles of Chief Risk Officer and Chief Credit Officer. Cary has been instrumental in guiding us through this transition, and we thank Cary for his service. Jim Crotwell will assume the role of Chief Risk Officer in 2019, and Preston Moore assumed the role of Chief Credit Officer earlier this year. Both Preston and Jim bring decades of banking experience to the roles, and will continue to guide these functions of the company. I'm extremely proud of the success of our company throughout the year and where we finished the quarter.
We crossed $7 billion in total assets, $1 billion in warehouse loans, posted record net interest income and record pre-tax, pre-provision income, expanded our margin, had another strong performance in mortgage banking and experienced stabilization and kept credit metrics across the loan portfolio. A special thanks to all our employees who worked tirelessly on a daily basis to deliver for our customers, shareholders and communities.
I'll open the call for questions.
Questions and Answers:
Operator
[Operator Instructions] Our first question today comes from Brady Gailey with KBW.
Brady Gailey -- KBW -- Analyst
Hey, Thanks. Good morning, guys.
Drake Mills -- Chairman, President and Chief Executive Officer
Good morning, Brady.
Brady Gailey -- KBW -- Analyst
I wanted to start with growth. You all had great loan growth, both on a core basis and in the warehouse. How do you think about growth going forward, kind of looking at those two segments, your core loans ex-warehouse, and then I mean the warehouse has just done so well. I mean, do you think it's stable from here? Do you think that you could see some downside from here in the warehouse?
Drake Mills -- Chairman, President and Chief Executive Officer
Yes, Brady, thanks. A couple of things here. And let me answer your first question. I think going into 2021. And we certainly see loan growth in the low to mid-single-digit area. A couple of things that's impacted growth at this point, line utilization, especially on our C&I, which I think creates confidence is down $167 million from the quarter. We also -- just the migration in energy, where we started out the year at $128 million, and now it's down to $55.5 million, is a natural progression in a down market. Certainly, it's worked like it as opposed to mortgage warehouse has just been a big win. But -- and I think what -- if you look at -- and I talked about this, I think, last quarter. If you look at our numbers based in a normalized environment when the market slows down, we have outstanding of $200 million to $300 million. We've taken, I think, strong advantage of dislocation in the marketplace.
Our team has done extremely well. We have picked up significant relationships in the last two quarters to bolster our numbers from running about 20 clients to close to 40 clients. So we see that number normalized once things slow down, potentially in the $500 million to $600 million range and probably see mortgage warehouse finishing the year in the $750 million to $800 million range. So it's holding up well for us. And so we see some -- we have good pipeline activity right now. Certainly, our C&I companies are starting to show signs of confidence. So we just think we'll get back on track, certainly not to the higher end of the single digits and our 10%, but certainly in that low to medium range, single-digit range.
Brady Gailey -- KBW -- Analyst
Right. That's helpful. And then it was good to see the reserve go a little higher in the quarter. I know you've kind of been talking about this two to 2.25 range, where you're basically there now. So do you think the reserve build is over? Or do you think you continue to add a little bit to that over time?
Drake Mills -- Chairman, President and Chief Executive Officer
Yes, it's going to be interesting this quarter. We've got certainly mixed signals, and we're paying credit deterioration and certainly any type of migration based on industry deterioration or anything. So the numbers, the increases that is going to have to come from our model and the metrics around that model. And I think you have to look at the fact that we do see some renewed confidence, let's say, in the market. But on the other hand, you have a potential change in administration, COVID peak back up and some lockdowns and those type of things, certainly kind of fog your thoughts about what that looks like. So I'm still saying that we could potentially see a slight build in the fourth quarter, which at that point, I would say, which currently wrap us up, and I don't see that number going to 2.5 in any way, but certainly could split that number between where we are today and 2.5 if the model supports it.
Brady Gailey -- KBW -- Analyst
All right. And then finally for me, in other conference calls, this earnings season, we've heard some CEOs talk about a little uptick in the M&A conversations. I was just wondering if that was the same for you guys. And just kind of what your take on M&A was at this point?
Drake Mills -- Chairman, President and Chief Executive Officer
I wouldn't say an uptick. Certainly, M&A is a focus of ours today. We think for us to really perform the way we desire to perform. It would be extremely helpful to have some partners in the process, especially around strong the deposit franchises. So we continue to have those conversations. And certainly, that is a desire of ours, but I don't see at this point, I could say that there is an uptick in it. But we continue to -- we're dealing with multiple opportunities, and it's a strong desire of ours to stay active.
Brady Gailey -- KBW -- Analyst
Great. Thanks for the color guys.
Operator
Our next question will come from Matt Olney with Stephens.
Matt Olney -- Stephens -- Analyst
Thanks, good morning guys. I want to circle back on the mortgage warehouse commentary, and Drake, you mentioned on-boarding some new customers. Any more color you can add as far as how many customers you've added? I think you mentioned, in normal times, the range was 200 to 300 and now you're saying normal times would be, call it, 500 to 600. So are you saying that you've doubled the number of customers in the warehouse more recently?
Drake Mills -- Chairman, President and Chief Executive Officer
Yes. We -- I'm not going to use round numbers here. Probably this time last year, we were dealing with 2021 customers. Today, we have 36. We're about to onboard four additional customers. But what's been interesting about the top -- this next 20 that we're adding is that they seem to be larger clients, a lot more stable, slightly stronger and better core deposits from the standpoint of non interest-bearing deposits. So that's why I'm up in this based on our belief of what utilization is going to be.
Matt Olney -- Stephens -- Analyst
Okay. And Drake, sticking on the same topic on the warehouse, what about the associated cost with adding these new customers? Are you adding new people, new systems? Just trying to figure out the operating leverage on these wins?
Drake Mills -- Chairman, President and Chief Executive Officer
Matt, that's a big win for us. We are extremely efficient there. Ken and Jason Johnson drive that team. Our backroom team is driven by a lady that is completely awesome. I think we've added one additional person through this process. We don't see a buildup in expense at all. That's why it's a huge win for us.
Matt Olney -- Stephens -- Analyst
Okay. Great. And then circling back on loan growth ex-warehouse and ex-PPP, it sounds like that the levels you achieved in the third quarter maybe the high watermark, maybe tough to achieve that again. Can you just talk more about what we saw in the third quarter, the strength there? Was it from slower paydowns that we've seen that could accelerate or originations higher than expectation? Just any more color on the 3Q growth?
Drake Mills -- Chairman, President and Chief Executive Officer
Yes. No, it was origination. We saw strong growth in residential and CRE, and I really think looking at pipelines, we could see that -- a similar outcome for the fourth quarter based on what we're seeing and strong activity in the market. It's just, obviously, a little more difficult to underwrite. There's some areas that we're not willing to expand the portfolio we in. So we're staying true to our strategy since COVID, but I do see somewhat of a repeat of the fourth -- the third and the fourth.
Matt Olney -- Stephens -- Analyst
Okay. And then just lastly on capital. You mentioned in your remarks that you raised some capital a few weeks ago. I'd love to hear more thoughts about deploying that capital and specifically thoughts about the buyback program. Just remind us what the current amount of authorization is. And what's the appetite to activate that buyback program?
Lance Hall -- President and Chief Executive Officer
Yes. And I'd like to back up a little bit because this goes back to a strategy, let's say, from 2019, when we really created a dividend strategy that would make our stock more attractive from a yield perspective. So each year, we'll continue to analyze where we are from a dividend perspective as we did this quarter. We'll do that next year and felt that we needed to continue that strategy, which really what that did was allow us to, I think, with the strong capital position we have and the confidence we have in the portfolio, where we see 2021 performance come, we just felt like that dividend needed support, which runs into a buyback. We do have -- we -- in July of 2019, we established a $40 million buyback program, certainly slowed that or suspended it in March. That has been on the table. And through this process, it was just basically a communication to the Board, hey, we need to start focusing on buyback opportunity.
And what I would say is that we've got $29.2 million remaining in that. It is a tool. It's on the table for us to use. We certainly, with the capital position and why the sub-debt came to be a focus was, when we look at total capital, certainly, we had strong Tier-one total capital. We didn't want to slow down our growth opportunities. We see some opportunities in 2021. The market was perfect, I think, for us to take advantage of adding some additional Tier-two capital, which what that did was really allowed us to focus on where we think those opportunities are continuing because we had planned on, at one time, seeing a steep pullback in mortgage warehouse, which would have actually refreshed that capital ratio, especially on the Tier-two side. But with that opportunity, we just felt sub debt could bridge that gap, give us the opportunity to stay in a gain from a loan growth standpoint and keep us focused on M&A. The unfortunate aspect is with our multiples, the way they are. It's tough to think that we can be significantly successful with high-quality M&A targets.
So this just puts us in a position to recognize that our stock is the best buy out there for us right now. So as we move in through the fourth quarter and continue to look at a couple of opportunities, I think it's going to be prudent for us to really get active, let's say, potentially in buyback if the market continues like it is.
Matt Olney -- Stephens -- Analyst
Okay. Great, thank you.
Operator
Our next question comes from Brad Milsaps with Piper Sandler.
Brad Milsaps -- Piper Sandler -- Analyst
Hey, good morning, guys.
Drake Mills -- Chairman, President and Chief Executive Officer
How are you doing?
Brad Milsaps -- Piper Sandler -- Analyst
Good. Final little blind day, no power, Internet in Atlanta. So bear with me. You guys know are used to that, though. So -- yes. Yes. So I'm preaching the choir. Just wanted to maybe ask quickly about the net interest margin. It looked from best I can tell that you guys did a pretty good job of holding kind of loans sort of ex-warehouse, ex-PPP what else ever you want to throw in there, pretty stable or maybe down slightly linked quarter. Can you talk about your ability to continue to do that and sort of kind of how that relates to kind of what you're seeing on -- in terms of the net interest margin as you look out into 2021?
Drake Mills -- Chairman, President and Chief Executive Officer
Yes. I think our team is doing awesome job, especially on the deposit side. We continue to feel like we have some room to go, reducing deposits in an extremely competitive environment on the loan pricing side that takes away some of that strength. But we do feel we had some benefits of, I think, pricing strategy, especially on the deposit side that drove our NIM expansion this quarter. I do want to remind everyone that with the sub debt, we will probably see a five basis point impact to the margin in the fourth quarter and feel that with that impact is where we see things at this point that our margin for the fourth quarter could look a lot like our second quarter margin, if I could say that, which we feel very comfortable with. So at this point, a pullback because of sub debt maybe a basis point here or there but very similar to what we saw in the second quarter, and then hopefully, we'll start seeing some wins in 2021.
Brad Milsaps -- Piper Sandler -- Analyst
Great. That's very helpful. And then just as a second follow-up, maybe for Steve, just on the retail mortgage banking business. I know the second quarter was impacted by a big positive swing in the MSR. Just kind of curious what that impact was in the third quarter. I know that will be in the 10-Q, but just kind of curious what that was. And then kind of how your gain on loan sale margin compared to the -- in the third versus the second quarter.
Stephen Brolly -- Chief Financial Officer
The gain on sale was higher this quarter, and we had a little bit less on volume on originations and sales. The MSR did not have an impact -- a large impact at all. It was pretty much neutral, maybe a couple hundred thousand dollars down. So that's really why we went from ten down to nine. And if you're expecting MSR to go a little lower, that nine would have been maybe seven or eight, but the MSR really kept pretty much in line. I mean, after Q1, after March 31, that really MSR came down with the market. And since then, it hasn't changed dramatically.
Going forward, in the fourth quarter, we expect about $150 million to $170 million in production and sales. We had about $220 million this quarter. So the fourth quarter historically is not as strong as the second and third, and that's what we're expecting next quarter also.
Brad Milsaps -- Piper Sandler -- Analyst
Okay. Great. No, that's really helpful. And just a follow-up to that. Drake, I know you guys spent the last couple of years making a lot of changes there, trying to get more efficient. Obviously, a great year this year with refis and rates going lower. How much more market share do you think you can gain in kind of 2021 kind of relative to what you did this year? Just trying to get a sense of what the impact will be as eventually mortgage does start to contract a little bit. Just kind of curious kind of what you're thinking around that business.
Drake Mills -- Chairman, President and Chief Executive Officer
Yes. I think we can continue on the pace from a market share standpoint like we are because of really lift out teams in Mississippi and Louisiana. Especially in our Monroe market, that's done extremely well. We have some of our two top producers are in Ruston and Monroe that are running in that $8 million to $10 million a month. So we still have opportunities. Our Houston team is looking at expanding the mortgage origination side. So I think we can continue to win like we have this -- obviously, with volumes down, I think, this quarter, I mean our origination really showed strong. They come in about 50% of total volume, and we still see pipeline strong. So at this point, I just think market share will continue to come our way.
Brad Milsaps -- Piper Sandler -- Analyst
Alright. Thank you, guys.
Drake Mills -- Chairman, President and Chief Executive Officer
Thank you.
Operator
Our next question comes from Kevin Fitzsimmons with D.A. Davidson.
Kevin Fitzsimmons -- D.A. Davidson -- Analyst
Hey, good morning, everyone.
Drake Mills -- Chairman, President and Chief Executive Officer
Good morning, Kevin.
Kevin Fitzsimmons -- D.A. Davidson -- Analyst
I'm just curious on credit, drilling down a little further in terms of -- I noted the impressive decline in deferrals and what you expect that to continue. I'm curious about loans, any migration you're seeing to the watch list to special mention to substandard, whether any of those former deferrals or remaining deferrals or any other loans, for that matter, notable migration that you're starting to see. Thanks.
Drake Mills -- Chairman, President and Chief Executive Officer
As we report, I think there's -- we're seeing a stabilization. And I would say at this point, it's not deferrals that are migrating to a large degree. I thought it was kind of interesting that I heard a CEO kind of take a slap at banks that we're aggressive with deferrals and saying that it must have been a feel good moment. But I can tell you for us, as a company that focuses on small businesses and C&I firms, that this is real. It's not a feel good moment at all. It's supporting these companies. And the way we approach this is if they were in good standing and we're a company that could move forward, we certainly are going to continue to support them. We don't play games. Now we addressed some of these companies, and we said that last quarter that we had problems, we're going to purge ourselves of these things.
We're very aggressive with that. But it is a focus of mine and like I said, not a feel good moment to do everything I can to keep these companies focused on being viable and continue to go on once you return to some type of normal environment. So -- but I'm going to let Jim Crotwell for just a minute kind of walk through some of the migration and what components are. Jim?
Jim Crotwell -- Chief Risk Officer
Thank you, Drake. Good morning. Really, we've seen very strong resiliency in our portfolio. As you can see on slide 16 that our overall level of classifieds and special mention loans have been very stable over the last two quarters and really no change. And as we look forward, we continue to anticipate that stabilization in our total criticized asset base. Obviously, during this pandemic, we've been proactive in the pass watch category and just the pass category as we look to internal risk ratings perspective. And as mentioned in the presentation, all of our internal risk ratings were validated by our annual external loan review. So right now, we're seeing a good bit of stability within our portfolio, and we do not anticipate any significant shifts.
Kevin Fitzsimmons -- D.A. Davidson -- Analyst
Okay. Great. That's helpful. One thing I just -- as a follow-up, I want to go back to, I just want to make sure I'm understanding it correctly. The mortgage warehouse lines of credit, which were a little over $1 billion on an end-of-period basis and about $724 million on an average basis in the quarter, Drake, will you -- your comments -- commentary was that in a normal environment, that runs from $200 million to $300 million per quarter. And you're saying that because of the pickup in new customers, that's going to normalize out at more like $500 million to $600 million. And then you commented that, by the end of the year, you could see speed up that number being $750 million to $800 million. So with the slowdown, does that mean we're going basically from the $1 billion down to $500 million, $600 million over the next few quarters, but then it ramps up from there and the year where you said? I just want to make sure I'm understanding that correctly.
Drake Mills -- Chairman, President and Chief Executive Officer
And sorry if I created a confusion there. I said a lot in a very short period of time there that in a normalized environment, we would run $200 million to $300 million with the customer base that we have, with the dislocation market and our ability to take advantage of that in a normalized environment with our current customer base on an -- on a forward going slowed market, that would look like $500 million to $600 million. My comment about $750 million to $800 million is where we think we'll end up at the end of the fourth quarter. But as we move forward in 2021, we'll start to see a slowdown. And like I said, comparing that to a normalized environment pre 2020, that now looks like $500 million to $600 million outstanding.
Kevin Fitzsimmons -- D.A. Davidson -- Analyst
That's great. That cleared that for me. Thanks Drake.
Drake Mills -- Chairman, President and Chief Executive Officer
[Indecipherable] I don't get a good job on that.
Kevin Fitzsimmons -- D.A. Davidson -- Analyst
No, no, that was on me. One last one for me. Just the swap fees, that was -- went from a very -- quite a high number to a low number this quarter. And just where you guys see that going forward?
Stephen Brolly -- Chief Financial Officer
Kevin, last quarter was one of the highest we've had, and so we normally -- look at it and say, maybe 100,000 a month, sometimes 200,000 a month. And last quarter, it was just the second highest we ever had. And so this quarter is a little bit lower, but I think going forward, it would be -- if you took both quarters and average them, that would be a good place to at least estimate where it would be going forward.
Kevin Fitzsimmons -- D.A. Davidson -- Analyst
Okay, thank you everyone. Thanks.
Drake Mills -- Chairman, President and Chief Executive Officer
Thank you.
Operator
[Operator Instructions] Our next question comes from William Wallace with Raymond James.
William Wallace -- Raymond James -- Analyst
Good morning, guys.
Drake Mills -- Chairman, President and Chief Executive Officer
How are you doing?
William Wallace -- Raymond James -- Analyst
Very good. Thanks Drake. First question on the expense line. Steve, in your prepared remarks, you said that you think you could hold the line at $37 million to $38 million. Did you mean that you think that you could hold it there for 2021 or for the fourth quarter and then we would expect kind of normal seasonal pressures in 2021?
Stephen Brolly -- Chief Financial Officer
That would be for both. That would be for where the budget is right now for next year.
William Wallace -- Raymond James -- Analyst
Okay. And how does that contemplate mortgage activity and all the variable comp associated with that?
Stephen Brolly -- Chief Financial Officer
That would be -- the mortgage activity would be a little bit less than last year, on an average, just because of all the refi we had in the second and third quarter. But I said, about $150 million to $160 million next quarter, and we did about $220 million. So we think the first quarter may be somewhere in the low 100s, and we'll be back up to closer to 180 million, 200 million third quarter next year. So that does take into effect the compensation for the mortgage originators, but we also had a lot of compensation this year also.
Drake Mills -- Chairman, President and Chief Executive Officer
Yes. Wally, if I could, I'd like to add to that just for a moment because we have -- as we've discussed, Lance Hall is extremely active with his team in looking at expenses across all lines and then some real strong moves, and we'll continue that to 2021. The offset to that is as we still have confidence in our markets, we've had, as I noted in the second quarter, a great opportunity in Dallas and another office to really what we think has created a core deposit growth strategy that's significant for us in that market. So we'll have some added expense there. We also, in the Woodlands and Houston, as we've communicated, we'll be moving offices that -- and picking up and ramping up from a lending standpoint some opportunities there. So we're baking all that in and really trying to offset that with some strong management on the expense side. So we still see opportunity for growth in this organization, but we feel like it's necessary to be very active on the expense side to be able to cover that.
William Wallace -- Raymond James -- Analyst
Okay. Thank you. That's helpful. Drake, on net interest margin, you said, you think fourth quarter, we probably see margin back down to roughly where it was in the second quarter, but then that you would get some wins, I believe, is what you said from there. Does that mean that you expect that margin could start to expand next year? And if so, is that just a deployment of liquidity? Or do you feel like you're bottoming out on the loan yields and deposit costs moves will help drive expansion?
Drake Mills -- Chairman, President and Chief Executive Officer
You know certainly see opportunity there. Bottoming out of deposit cost is certainly going to be a headwind for us because we have, I think, are some significant pushes through incentives to grow core deposits next year with that being more of a focus than on the loan side. So we're taking those teams to be able to are the focus that I think where the concern is for us at this point is where the competitive side of loan yields come in because we're still seeing some pretty, some pricing that's tough to handle at this point, but we're going to stay focused on our customers. So really, I think the win is if we can potentially focus on that is the utilization of our cash and being able to put liquidity to work, which is some strategies that we'll be working on through this quarter.
William Wallace -- Raymond James -- Analyst
Okay. Thanks. And then last question, just kind of bigger picture, Drake. Gosh, a year ago, there was a very myopic focus on deposit growth, core deposit growth at the bank. Obviously, stimulus and PPP funds have muddied the picture. And it looks like this year or this quarter, you put some brokered deposits on to replace some other funding. Can you just talk maybe what's going on at the bank from a core deposit perspective? Do you feel like you're having successes? I just look at that loan-to-deposit ratio. I don't know what it would be if we took the brokered deposits out. But do you feel like your -- that focus a year or so ago is reaping rewards if you kind of peel back some of the impact of the stimulus checks and PPP dollars that have come on to the balance sheet?
Drake Mills -- Chairman, President and Chief Executive Officer
No, absolutely. I mean if you look at our markets and the growth markets year-to-date from core deposit growth is pretty phenomenal. We're seeing some unbelievable wins. And as I said earlier, it drove us to take advantage of an opportunity in Dallas that we think can add a couple hundred million dollars of core deposit growth over the next 24 months. That is significant for us. So what we're doing is really shifting our teams from an incentive compensation standpoint, more to core deposit growth in a period of time where we see the opportunity to take advantage of the market, as we see a slowdown on the loan side.
So if you normalize our loan deposit ratio, taking out all the noise, I would say we're running around 88%, 89% and feel pretty good about where we are, but understand that as we have this slowdown in loan activity, we see strong opportunity in all markets to win on the core deposit side. So our focus in 2021 is going to be core deposit growth.
William Wallace -- Raymond James -- Analyst
Great. Thank you for that Drake. I appreciate it. I'll step back.
Operator
[Operator Instructions] Our next question is a follow-up from Matt Olney with Stephens.
Matt Olney -- Stephens -- Analyst
Yes. Thanks for taking the follow-up. Just want to circle back on this discussion on the margin and specifically on loan yields. Loan yields came down a bit in the quarter, but it was pretty modest. Any commentary on how you see the loan yields playing out over the next few quarters? And I guess I'm looking for any color on recent loan growth, if you have the new and renewed loan yields more recently. I just want to figure out how much more pressure we could see there. Thanks.
Drake Mills -- Chairman, President and Chief Executive Officer
Yes. This is -- I think in an environment, and I know why you're asking where you're coming from, that we see some headwinds on the loan yield from a loan yield aspect and continue to think that it's going to take balance sheet management and a couple of other things to be able to thin that out because of where we see deposit costs start to slow down from a decline. So moving forward, we're going to -- and I will say this, we're going to create -- and I keep talking about incentives and look, incentives drive everything we do. And we're today working on a method to be able to incent and reward our relationship managers based on yield and their ability to get yield, very, very active and going to almost require floors in any situation.
So I think that, that's going to be the real opportunity for us to slow down or kill some of the headwinds coming from a loan yield perspective. So you're going to see strong incentives. That's going to, I think, drive better pricing. You're going to see floors. That's certainly going to slow down this situation. I think we can do that in a very competitive environment and still see the loan growth I'm talking about.
Matt Olney -- Stephens -- Analyst
Okay. And I guess...
Drake Mills -- Chairman, President and Chief Executive Officer
Matt, I would say this, the big win for us on the deposit side is really going to be where CD pricing is right now and where we see CD pricing heading. That's the next trigger to pull and where we see a win at this point.
Matt Olney -- Stephens -- Analyst
Okay. Great, that's all for me. Thank you, guys.
Drake Mills -- Chairman, President and Chief Executive Officer
Thank you.
Operator
This concludes our question-and-answer session. I would like to turn the call back over to Drake Mills for any closing remarks.
Drake Mills -- Chairman, President and Chief Executive Officer
Okay. First off, thank you, everyone, for the opportunity and support. It's pretty incredible how comfortable I am at this point and what we're doing, the confidence that we have in this organization, as I said in my closing remarks earlier. But I will say that thanks to the support and thanks to you that people inside this institution, that our culture is stronger. Our attitudes are incredible. And that was rewarded here in the last couple of days when our American Banker released their top banks to work for.
And it was such a pleasure to be voted in the top 20 of the best banks to work for. So that shows that we're winning in a very difficult environment. Our culture is stronger. Our people feel supported. They feel safe. And all those things are going to end up and be rewarded back to the shareholders in return. So I do appreciate in a huge way your confidence and your support. Thank you.
Operator
[Operator Closing Remarks].
Duration: 53 minutes
Call participants:
Chris Reigelman -- Investor Relations
Drake Mills -- Chairman, President and Chief Executive Officer
Lance Hall -- President and Chief Executive Officer
Jim Crotwell -- Chief Risk Officer
Stephen Brolly -- Chief Financial Officer
Brady Gailey -- KBW -- Analyst
Matt Olney -- Stephens -- Analyst
Brad Milsaps -- Piper Sandler -- Analyst
Kevin Fitzsimmons -- D.A. Davidson -- Analyst
William Wallace -- Raymond James -- Analyst
