Image source: The Motley Fool.
Cadence Bancorporation (CADE)
Q4 2019 Earnings Call
Jan 23, 2020, 8:30 a.m. ET
Contents:
- Prepared Remarks
- Questions and Answers
- Call Participants
Prepared Remarks:
Operator
Welcome to the Cadence Bancorporation Fourth Quarter and Full Year 2019 Earnings Conference Call. All participants will be in listen-only mode. The comments are subject to the forward-looking statement disclaimer, which can be found in the press release and on Page 2 of the financial results presentation. Both of those documents can be located in the Investor Relations section at cadencebancorporation.com. After today's presentation, there will be an opportunity to ask questions. Please note that this event is being recorded.
I would now like to turn the conference over to Paul Murphy, Chairman and CEO. Please go ahead.
Paul B. Murphy -- Chairman and Chief Executive Officer
Well, good morning and thank you for joining our earnings conference call. With me today on the call are Valerie Toalson, Sam Tortorici, Hank Holmes, and David Black. For 2019, Cadence had several key accomplishments to report and we delivered attractive returns despite elevated credit costs. Adjusted earnings for the year were $223 million, up from $175 million in '18. For the year, this was a 1.3% ROA, 13.6% return on tangible common equity, and a 49% efficiency ratio, which is an improvement from 50% last year. Many banks would consider these to be pretty good numbers, but I do not -- we do not. I would say however that posting these numbers in a year marked by elevated credit costs is a noteworthy achievement and is a testament to the underlying earnings power of the franchise.
A key earnings strength indicator we look at and the Board look at is pre-tax, pre-provision earnings, PTPP. This increased from $242 million in '18 to $400 million, up 66% linked Q. PTPP to average assets went from 2.1% in '18 to 2.26% in '19. This progress is due to the successful State Bank merger, continued business development efforts, good expense control, and NIM outperformance. With the normal provision run rate, this level of PTPP will generate attractive returns for investors.
In 2019, our capital ratios strengthened across the board compared to prior year. This is a nice accomplishment, especially considering the share buyback of $79 million, $90 million in dividends to shareholders and we reduced debt by $50 million. Tangible book value ended the year at $14.65 per share, up 8% from the prior year. NIM improved significantly due to the merger and effective balance sheet management, especially including the hedge.
I'd like to call to your attention a key part of the core NIM results for the fourth quarter, which is that our loan yields were down 9 basis points linked quarter, while deposit costs were down 18 basis points. Valerie is going to review our NIM solid performance in more detail in her presentation.
Next, let's talk about deposits. 2019 was a great year. We've been able to transform and improve the mix and quality of the deposit base. For the year, core deposits grew $780 million or 8% excluding State. Our brokered deposit portfolio has been dramatically reduced from a peak of over $1 billion or 10% to today less than $200 million or [Phonetic] 1.3% at fiscal year-end and note that deposit costs are down 20 basis points over the fourth quarter of '18 and are down 18 basis points linked quarter. This is largely driven by the growth in our non-interest bearing core deposits, which are now 26% of the mix versus 23% prior year. Our liquidity also improved. Our loan to deposit ratio now stands at 88% versus 94% a year ago. Our dedicated bankers have been very focused on growing core deposit relationships and the State Bank merger contributed significantly to these nice results.
In terms of expense management, we're pleased with the year as our adjusted efficiency ratio dropped nearly 100 points to 48.6%. We have a cost-effective model today. However, we're never satisfied. Cost management remains a key focus of the team. We have been and will be making some offensive hires in Georgia, Dallas and other parts of the Company, and we expect to see nice returns in future years from the investments we will make in 2020.
To comment a bit further on State Bank merger, first I have to say again how proud I am of the State Bank bankers and the leadership that they have shown of what is now almost two years of working together. Clearly, we're very pleased with the earnings accretion and our ability to realize the expense saves of the merger. Importantly, we're very optimistic about the strategic benefits of adding key revenue producers in Georgia.
Another point that might make -- might interest investors is that our Board merger has gone rather well in my opinion. State Bank directors and new directors are fully engaged. We can feel the chemistry and the diversity of the Board is very healthy at this time. Our management team and Board are committed and aligned with our shareholders. Over 2019, our combined management and Board made open market purchases of over $7 million worth of shares and we collectively own 4.2 million shares or 3.25% of the company.
Shifting to credit. Cadence incurred $86 million of net charge-offs or 63 basis points for 2019. Obviously, I'm very disappointed with these results and I take full responsibility. Today, I strongly believe that we're taking the appropriate action to remediate risk in our portfolio and the credit costs will not be a long-term detractor to the value of the franchise. We're recognizing credit downgrades promptly, we're addressing the issues, and we're moving forward. To frame up the credit discussion bit further, I think it will be helpful to disaggregate these charge-offs by key loan categories and then describe a few themes and characteristics across each. First, let's look at our full-year charge-offs by loan type, C&I credits. $44 million or 51% of the total were six general C&I loans that included one outlier of a $20 million charge-off, a very high severity loss credit, a non-SNC credit. The $44 million to the -- relative to the $4 billion general portfolio is a charge-off rate of 1.1% for the year. Restaurant was next with $21 million in charge-offs. This is about 24% of the total, is five credits. It's a 2% charge-off rate on the $1 billion Restaurant portfolio. And the last three Energy borrowers accounted for 15% of the charge-offs for the year. It's $13 million in dollars and it's 1% of the Energy book. These Energy charge-offs were from credits originated prior to and in 2014. And as we previously reported, we changed our E&P underwriting guidelines and our engineering approach at that time. So our post-2014 E&P portfolio has performed rather well.
So, I believe the Restaurant portfolio remains the highest risk portfolio in the bank. The key issues there are margin and labor pressure, some overdevelopment, and third-party delivery. Several of our borrowers are on a viable path to be upgraded as a result of positive improvement in operating results and some are going to be able to refinance and reduce exposure. Others are more challenged and a path to an upgrade is [Technical Issues]. I'm sure our frontline bankers and management team is highly engaged with this portfolio and we're actively working with clients to shore up their operations. At 12/31, the portfolio balance was $993 million, down from a peak of $1.25 billion. During the fourth quarter, the Restaurant portfolio declined by $57 million.
The next highest area of risk would be Leveraged loans without a moderator and as similarly with the Restaurant, we are diligently working through this portfolio to improve the risk profile or exit loans where we're not comfortable. As a result of this work in the last six months, we've reduced the size of the portfolio by $183 million from a peak of $875 million down to $692 million at year-end.
On a more positive note, let me turn to a number of business units that are reporting very good credit results. These portfolios in aggregate make up the majority of our total loan portfolio. Let's review the history of them over the past three years since going public. So, energy services and healthcare combined are about $655 million of total loans. They've experienced zero charge-offs over the last three years. The legacy Cadence $1.5 billion commercial real estate portfolio has an aggregate charge-offs of $16,000 over the last three years. So, I would call that outstanding results and you can probably call that zero charge-offs. The $2.8 billion commercial real estate portfolio from State, net charge-offs for 2019 were below $4 million, 14 basis points. This would make their three-year charge-off average around 5 basis points. The $2.1 billion residential real estate portfolio at Cadence is pristine and has recorded charge-offs of less than $500,000 since 2017. And last, touching based on the technology portfolio of $388 million had one charge-off of little less than $0.5 million or 12 basis points in 2019 and their three-year average would be closer to 4 basis points.
So to summarize, 2019 net charge-offs of 63 basis points brings our three-year average from our originated loan portfolio to 31 basis points, which we would say is in an acceptable range for our model. I expect to see 2020 be an improved year for credit. Restaurant and Leverage without moderators have elevated risk, that's for sure, but I believe the rest of the portfolio will continue to post good credit results and I do not foresee another year of 63 basis points in charge-offs.
So, OK, Paul, specifically, why do you think 2020 will be better? Well, the portfolio has been de-risked in several ways. Our exposure to leverage loans without moderators has declined $183 million or down 21%. The Restaurant portfolio was down $250 million or 20% from the peak. Our Energy portfolio is predominantly midstream and given the recent recovery in oil prices, we believe the risk in this portfolio are reduced. And as I mentioned, our E&P portfolio underwritten post-2014 has performed well. Energy non-performers today are less than $10 million and we have one energy credit of little less than $20 million that's on our exit list. So clearly, energy capital market is a challenge, I get that, but we feel pretty good about where we stand today.
So, as I look back and analyze our recent credit migration, we see the ramp up in criticized and classified assets have slowed relative to the uptick that we had in the third quarter. It's our expectation that criticized and classified loans will begin to decline over the first half of 2020 as we see some relationships active, we see some upgrades due to operating performance and some paydowns from refinancing. Hopefully, this information is useful and we're making every effort to be as transparent as possible with how I see the credit situation. So again, our model generates attractive returns at 25 basis points to 30 basis points in charge-offs. At 63 basis points, I'm not happy, it's not acceptable, but 63 basis points is manageable. The loan portfolio declined 5% linked quarter, driven primarily by paydowns in Energy, Restaurant and Leveraged. So for 2020, we now expect our loan growth starting from the 12/31 period to be in the 3% to 5% range for the year. Our new business pipelines were down from prior years but are healthy and they support the mid-single digit growth targets that I just mentioned. As a reminder, I'm really pleased and proud of the highly motivated team of bankers that are working hard every day to do a great job for clients and grow the business.
So with that, let me turn the call over to Valerie.
Valerie C. Toalson -- Chief Financial Officer
Thanks, Paul. Adjusted net income for the fourth quarter of '19 was $51.9 million, with an adjusted EPS of $0.40, up $7.8 million and $0.06 per share respectively from the prior quarter. For the full year, adjusted net income was $223 million, up from $175 million in 2018. Adjusted EPS for the full year 2019 was $1.72, down from $2.07 from the prior year. Adjusted ROA for the fourth quarter was 1.16% and the full year was 1.26%. As Paul mentioned previously, we continued to experience strong growth in core deposits, increasing $270 million or 1.9% from the prior quarter to $14.5 billion and we further reduced brokered deposits by $317 million to 1.3% of total deposits. For the full year, excluding the impact of deposits acquired from State Bank, core deposits increased $780 million or 8.1% and brokered deposits declined $842 million. At the same time, our loan balances in the fourth quarter declined by $653 million or 4.8% to $13.1 billion, due primarily to greater net paydowns and payoffs as we work to reduce portfolio risk, as well as $96 million in loans sold or moved to held for sale.
For the full year, excluding the impact of loans acquired from State Bank, loans decreased $387 million or 3.9%, including a $124 million in loans sold or moved to loans held for sale. As a result of this added liquidity throughout the year, we increased our investment securities mix to $2.4 billion or 13% of total assets, adding $663 million in the fourth quarter and increasing $514 million for the full year. This mix shift led to total revenue in the quarter being flat at $195 million with net interest income increasing $711,000 and non-interest revenue declining $744,000.
For the full year, total revenue was $782 million. Focusing on net interest margin, we are very pleased with our net interest margin dynamics this year and into the fourth quarter. For the full year, our NIM increased to 4% from 3.61% as our yield on originated and ANCI loans, excluding accretion, increased 34 basis points during the year to 5.38% and our cost of funds increased only 14 basis points during the year. The addition of the State Bank balance sheet, the implementation of our $4 billion LIBOR collar in the first quarter of 2019 and our targeted focus on funding costs all contributed to this margin improvement. Additionally, in the fourth quarter of '19, we were able to completely offset the impact of declining interest rates on our variable rate loans through our hedging activity and aggressive management of our cost of deposits.
In the fourth quarter, we saw an 18-basis point improvement in our deposit costs and only a 9-basis point decline in originated and ANCI core loan yields, excluding accretion. Our fourth quarter total deposit beta was 39% and our originated loan beta was 15%. As a result, NIM in the fourth quarter was essentially unaffected as a result of changes in yields and costs with the impact of lower LIBOR rates on our loans fully offset by one, materially improved funding costs; two, greater revenue from our hedging activities as LIBOR declined; and three, increased loan fees associated largely with the loan payoff activity. The 5-basis point decline in the fourth quarter NIM to 3.89% was actually attributable to 2 basis points decline as a result of the balance sheet mix, with the lower-yielding securities replacing higher-yielding loans, partially offset by lower-cost core deposits replacing higher-cost brokered deposits. A 2-basis point decline was due to the increased impact of non-accrual loans on the margin and 1-basis point decline due to the total accretion being lower during the quarter. Going to 2020, we continue to feel very good about our ability to maintain stability in our core margin in the forecast rate environment.
Noninterest expenses were also well managed for the year, as we integrated State Bank and fully realized the efficiencies we modeled for the combined companies. Our full-year efficiency ratio for 2019 was 48.6%, further improved from 49.6% in 2018. For the fourth quarter, the combination of the flat revenue and increased noninterest expenses resulted in our adjusted efficiency ratio up slightly to 50.9% for the fourth quarter from 49% in the prior quarter. Our 2020 expectation for the adjusted efficiency ratio remains in the low 50s.
Adjusted noninterest expenses for the full 2019 year were $379 million and for the fourth quarter were $98.4 million or up $5.1 million from the prior quarter. Approximately $3.3 million of the quarterly increase was the result of third quarter credits that lowered third quarter expenses in FDIC costs, incentive accruals, and loan costs and $1.8 million, due largely to increases in routine compensation, professional fees and advertising, and public relations expenses in the fourth quarter. While in 2019, we achieved the phase that we initially planned for the State Bank integration, we do expect to realize additional economies of scale in our operating platform as we go forward. At the same time, we are also continuing to invest in our technology, infrastructure, and talent to support ongoing growth of the Company. So as a result, on a net basis, we currently anticipate 2020 adjusted expenses to increase very modestly in the low-single digit range over the fourth quarter '19 run rate.
At 12/31/19, our allowance for credit losses was $120 million or 92 basis points of total loans. As of CECL update, based on the fourth quarter '19 data, our estimated reserves would increase to between 1.45% and 1.50% or an increase of 55% to 65% from the year-end level. For the originated portfolio only, we would expect 40% to 45% increase in our reserve levels, with the greatest percent increase from our consumer and CRE portfolios, with a life of loan under CECL is much longer than the current loss emergence period. We continue to expect the day one phase-in capital impact at less than 10 basis points of total capital.
In the fourth quarter, we also continued to execute on our share repurchase program, repurchasing $9.8 million of common stock at an average price of $16.21. And we further enhanced our capital position through net earnings and lower risk-weighted assets. So while we did experience elevated credit cost in 2019, it is important to note that we also enhanced our balance sheet mix on both the asset and the liability side, we strengthened our capital position, while we also increased dividends and bought that stock at favorable prices, we maintained stability in our core net interest margin as rates increased and then declined, we continued our efficient operating platform, while effectively integrated a large and complex acquisition, we expanded our footprint to include a key new market, and we further added to the talented and driven employee base that gets it done.
In summary, we have a lot to be pleased about at Cadence Bank and look forward to 2020.
Operator, we'll go ahead and open it up for questions now.
Questions and Answers:
Operator
Thank you. We will now begin the question-and-answer session. [Operator Instructions] And today's first question comes from Brady Gailey of KBW. Please go ahead.
Brady Gailey -- Keefe, Bruyette & Woods -- Analyst
Hey, good morning, guys.
Paul B. Murphy -- Chairman and Chief Executive Officer
Good morning, Brady.
Brady Gailey -- Keefe, Bruyette & Woods -- Analyst
I wanted to start with the buyback. You're not buying back a ton of stock here, about 0.5% this quarter, the same as last quarter. I mean, you have TCE of almost 11%, the stock's at $120 million [Phonetic] in tangible. Why not get more aggressive on the buyback in 2020?
Paul B. Murphy -- Chairman and Chief Executive Officer
Yeah. Brady, it's something that we just have an ongoing conversation with our Board on and we're looking at a number of different variables when we make those decisions. I mean, capital is strategic and -- but it's definitely something that we'll continue to look at.
Valerie C. Toalson -- Chief Financial Officer
We've been opportunistic with those purchases in 2019. You can see that by the average price that we bought that back at. And I think that's going to be the approach as we go forward at least for the near term.
Brady Gailey -- Keefe, Bruyette & Woods -- Analyst
All right. And then the expense guide for 2020 of low-single digit creep. I just wanted to clarify, that's not year-over-year, but that's based on the 4Q '19 expenses annualized. Did I hear that right?
Valerie C. Toalson -- Chief Financial Officer
Yeah, that's right, that's right. And effectively that's really kind of a net number from making some investment for the future growth of the Company and then continuing to really be able to realize some efficiencies from being a larger organization. There is more of that to come and we're working on that.
Brady Gailey -- Keefe, Bruyette & Woods -- Analyst
All right. And then finally for me, I mean, accretable yield levels have been fairly consistent if you look over 2019 at around $18 million a quarter. As we look to 2020, I know that bucket is a declining bucket. How should we think about accretable yield levels at 2020?
Valerie C. Toalson -- Chief Financial Officer
Yeah. So, in our Table 3 in the press release, we have a table that breaks it down and the ANCI accretion will just gradually decline quarter-over-quarter as that portfolio continues to work -- to wind down. On the ACI piece, most -- that will transition to PCD under CECL and what you can expect is for that population of loans to have revenue of about half of what it's had, really I'd say over the last couple of quarters as we go forward on a quarterly basis and that's because part of that discount is effectively added to those balances as we enter into the CECL world.
Brady Gailey -- Keefe, Bruyette & Woods -- Analyst
All right. Got it. Thanks, guys.
Valerie C. Toalson -- Chief Financial Officer
You bet.
Operator
And our next question today comes from Jennifer Demba of SunTrust. Please go ahead.
Jennifer Demba -- SunTrust Robinson Humphrey -- Analyst
Thank you. Good morning.
Paul B. Murphy -- Chairman and Chief Executive Officer
Good morning, Jennifer.
Jennifer Demba -- SunTrust Robinson Humphrey -- Analyst
How much do you anticipate the loan portfolio will be de-risked further in 2020, Paul, and what does that imply for net loan growth this year?
Paul B. Murphy -- Chairman and Chief Executive Officer
Yes. Thanks, Jennifer. As we've mentioned in the comments, the Restaurant and leverage without moderators are things that I think the most about and we've seen $250 million reduction in Restaurant, $183 million leverage without moderators. I mean, the basic day-to-day work of loan committee, just every time we look at a credit that has higher leverage, we are scrubbing it extra hard, we're either working in a way to strengthen the structure and be comfortable or we're asking them to refinance and find a different lender. So, how do I put a pencil to what -- to answer your question, I mean specifically, I wish I could. I can just tell you that we're looking at anything that we think falls at the higher end of the risk spectrum and we're improving it or we're asking them to refinance.
Jennifer Demba -- SunTrust Robinson Humphrey -- Analyst
Okay. And you said you think it will be a better net charge-off year in '20 than '19. With the guidance over a cycle of 25 basis points to 30 basis points, are you anticipating that it's -- the net charge-offs this year are probably somewhere in between last year's result and that long-term goal?
Paul B. Murphy -- Chairman and Chief Executive Officer
I think that's fair. Hopefully at the lower-end of the range. I mean, a lot of it depends on what happens with the Restaurant portfolio in particular. So, I'd like to be realistic with expectations and we have a good portfolio and as we called out, there is a number of categories that are doing fantastic. What I worry about is something comes along that has a high severity of loss that I can't see today. Everything we know today have been racked up, so to speak, but that potential exists.
Jennifer Demba -- SunTrust Robinson Humphrey -- Analyst
Okay. Last question. I think it was announced this week that Krystal filed for Chapter 11. Do you guys have any exposure there?
Paul B. Murphy -- Chairman and Chief Executive Officer
We are in the credit, yes. And we think that our -- what we know that we've kind of appropriately marked our books with the bankruptcy in mind.
Valerie C. Toalson -- Chief Financial Officer
Yep, that was taken into consideration at year-end.
Paul B. Murphy -- Chairman and Chief Executive Officer
Yeah.
Jennifer Demba -- SunTrust Robinson Humphrey -- Analyst
Okay. What's the amount on that credit?
Paul B. Murphy -- Chairman and Chief Executive Officer
Our portion is around $10 million.
Jennifer Demba -- SunTrust Robinson Humphrey -- Analyst
Okay. Thank you.
Operator
And our next question today comes from Michael Rose with Raymond James. Please go ahead.
Michael Rose -- Raymond James -- Analyst
Hey, thanks for taking my questions. So, Valerie, just saw that the securities portfolio grew a little bit as we move forward. Obviously, some of the brokered deposits are getting near a trough level. Should we expect the securities book to remain relatively stable here? Or are you guys finding good opportunities out there to potentially grow it?
Valerie C. Toalson -- Chief Financial Officer
Yeah. So it's really a dynamic of our loan growth to be quite candid and our deposit growth significantly. This year we had such great deposit growth and honestly, we see good growth in our core deposits continuing. And so to the extent that that exceeds our loan growth, then yeah, I do believe that we'll continue to grow, I think, modestly over the near future anyway, but continue to grow that securities book. We did bring down the brokered deposits. We'll probably always have just a little bit of those to keep that channel open for volatility in funding levels in different quarters, but the core loan -- or the core deposits being able to bring those in at the lower cost is something that's very positive and we'll continue to work on that.
Michael Rose -- Raymond James -- Analyst
Okay. That's helpful and maybe delving back to loans. So, Paul, I think at the outset, you said expectations are kind of 3% to 5% net growth this year. Can you just walk through where that's going to come from? Because if we think about, it looks like Energy is still going to be under some pressure, obviously, Restaurant as well, general C&I, there is some general softness in the market that you and others have mentioned. Can you just talk about kind of the puts and takes of that outlook? Thanks.
Paul B. Murphy -- Chairman and Chief Executive Officer
Yeah, Michael, I will and I'll ask Hank to comment also. So, yeah, we expect Restaurant to continue to come down. I think Energy will be pretty flat. I mean, we're open for Energy. It's a high bar, but we're feeling pretty good about where we are there. The C&I leverage thing, I mean, this is what makes it hard to predict with accuracy is what additional reductions what we have from anything that we feel is higher leverage, but we have 190 lenders, we're in some great markets, we're talented hard working bankers, they have attractive new business pipelines that are healthy, they're, again, not as high as some prior years, but we definitely have some good new business coming in the door, and we're going to be able to have some growth for sure, but I don't know with certainty is what will the payoffs be that kind of offset that. So, I don't know, Hank, how would you...
R.H. Hank Holmes -- Executive Vice President
Sure. No, I'll just add to that. Obviously, Michael, Sam has done a really good job of building out his team in Atlanta, which has seen some nice pipeline development there. We also had year-end, pretty much, probably 80%, 90% of our build-out in Dallas and so seeing some nice loan growth and deposit growth from both of those areas and then in our real estate and technology, healthcare and business banking, those folks are out throughout the footprint and we are seeing both the asset and liability growth from those areas as well.
One thing is we did have a high level of payoffs last year. However, we were still bringing in new clients, we are seeing early on in the first quarter that we may not have as many payoffs this year. It's a little hard to predict because with the non-bank lenders out there and what they do, but I'm optimistic about the first half of the year.
Michael Rose -- Raymond James -- Analyst
Okay. That's helpful. And then maybe just following up last one for me. On that expense outlook, can you give a little more color on the plans for hiring? We've seen several of your competitors talk about elevated costs related to hiring efforts from market disruptions and mergers and things like that, coupled with annual merit increases, inflation, stuff like that. So, I guess, what are the puts and takes of those investments and where you can expect to save some money on a go-forward basis? Thanks.
Paul B. Murphy -- Chairman and Chief Executive Officer
I'll just kick us off and just say we've made a lot of the hires in the Atlanta area and the Dallas area that we've kind of in -- that are in our budget and run rate. We are certainly going to be opportunistic where that exists with the teams we have in place are the ones that we feel good about where they are in their business development efforts. Sam, I might throw it over to you.
Samuel M. Tortorici -- President
Yeah, I'd say, that's very accurate. I mean, we pretty much have a lot of the hires in the Georgia market are already in our run rate and if we have a strategic opportunity here and there, we might take it, but largely the built out.
Paul B. Murphy -- Chairman and Chief Executive Officer
I might add that we still do add -- we still are adding treasury management support. So really continue that growth in the liability of the balance sheet as well.
Samuel M. Tortorici -- President
And Mike, we added about 10 production people in the Georgia market in the second half of the year and so that's in the fourth quarter run rate.
Valerie C. Toalson -- Chief Financial Officer
And just kind of take the other side of your question on where we anticipate having increased or -- efficiency, excuse me, is really as we look across the technology and the operations areas, those are the areas where we are putting in some things, looking at additional technologies, some targeted outsourcing efforts that will certainly allow us to realize some efficiencies there. Also looking at kind of some of our branch network modeling and some of the other staffing levels that we have throughout really kind of the operations areas that we believe there is opportunity. Takes a lot of time to get there. That's why we'll have some offset during the year.
Michael Rose -- Raymond James -- Analyst
Understood. Thanks for all the color. Appreciate it.
Operator
[Operator Instructions] Today's next question comes from Brad Milsaps from Piper Sandler. Please go ahead.
Brad Milsaps -- Sandler O'Neill + Partners, L.P. -- Analyst
Hey, good morning.
Paul B. Murphy -- Chairman and Chief Executive Officer
Good morning, Brad.
Brad Milsaps -- Sandler O'Neill + Partners, L.P. -- Analyst
Valerie, I just wanted to follow up on your comments around the margin. Obviously, a lot of moving parts. It sounded like you thought you could keep the core NIM stable. Just curious in the context, you'll be starting the beginning of the year with about $400 million or so lower in average loans and $400 million higher in securities, even with that you feel -- still feel good about being able to hold the margin steady as you kind of move through the first part of the year assuming rates remain stable?
Valerie C. Toalson -- Chief Financial Officer
Yeah, we really do, and part of that is due to our deposit costs and we've got about $1.5 billion [Phonetic] of CDs that are going to reprice over the first half of the year. They are at about probably 2.25% [Phonetic] right now, and so the replacement of those is coming in significantly lower. When you combine that with some loan growth as we go forward and obviously the impact of the hedge, we're pretty bullish on the stability of our margin.
Brad Milsaps -- Sandler O'Neill + Partners, L.P. -- Analyst
And the $700 million or so securities you bought in the quarter, can you give a sense kind of average rate of kind of where you're buying?
Valerie C. Toalson -- Chief Financial Officer
Yeah. Their duration is about five, five-and-a-half years and they're coming in probably about the $250 million [Phonetic] level. So they are lower than the loan yields, absolutely, and that's part of what you saw in the balanced mix shift, couple of basis points decline in the margin there.
Brad Milsaps -- Sandler O'Neill + Partners, L.P. -- Analyst
Great. Thank you very much. I appreciate the color.
Operator
[Operator Instructions] Today's next question comes from Matt Olney of Stephens. Please go ahead.
Matt Olney -- Stephens Inc. -- Analyst
Hey, great, thanks for taking my question. I want to go back to the loan balance contraction during the fourth quarter. We heard some commentary from you on the de-risking of the portfolio and then we also heard commentary about paydowns being elevated in the fourth quarter. Can you try to parse this down and give us some commentary about how much of the contraction in 4Q was from de-risking versus the paydowns?
Paul B. Murphy -- Chairman and Chief Executive Officer
Sure. I mean, first off, I mean it's a Restaurant, Energy, and Leveraged are the three categories that came down the most. I mean, there were some payoffs. We had some technology, great credits that we didn't want to lose that got paid off and refinance it non-bank lenders. So what's the breakdown exactly is your question.
Valerie C. Toalson -- Chief Financial Officer
Well, I think it's actually, I mean, there is kind of muddled together, right.
R.H. Hank Holmes -- Executive Vice President
Right. That's what I was going to say.
Valerie C. Toalson -- Chief Financial Officer
Yeah, I mean, some of the payoffs occurred because we were willing to change something, and therefore, they paid off and that was because of the de-risking move.
R.H. Hank Holmes -- Executive Vice President
Yeah, Matt, a lot of times what happens is as we have an acquisition or companies merge and they can get a higher leverage point, then we're willing to kind of chin that bar. A lot of times we keep the treasury, keep the working capital line, but maybe a term, debt or piece gets one that getting paid off. The non-bank lenders are pretty active in the second half of the year. So we saw that impact.
Matt Olney -- Stephens Inc. -- Analyst
Okay. I guess -- I guess that, but I'm trying to tie back to the 2020 loan growth guidance of 3% to 4%. If the de-risking is still going on, I guess, it's just hard for me to understand why the loan portfolio is going to grow 3% or 4% in 2020?
Paul B. Murphy -- Chairman and Chief Executive Officer
Because we've got good pipelines, we've got 190 lenders, we're out knocking on doors and the economy is good and throughout our footprint and there is a healthy view toward new business coming in the door.
R.H. Hank Holmes -- Executive Vice President
I agree with that and I also think the industry is looking at leverage from a cash flow perspective a little differently than it has historically. And so I think those numbers will be coming down.
Valerie C. Toalson -- Chief Financial Officer
And just to add to that and we've mentioned it before, that's a new market. It takes time to really build the momentum there and we feel like we've got some momentum behind our back.
Paul B. Murphy -- Chairman and Chief Executive Officer
But, Matt, I wouldn't want to try to convince you that it's in the can. I mean, it's definitely that's what we think, that's our estimate, 3% to 5% based on the pipelines and looking at what we think are likely reductions and de-risking. So that's our estimate of the net number, but it could be lower, hopefully it will be higher.
Matt Olney -- Stephens Inc. -- Analyst
And then on the Restaurant and the leverage book, is there a dollar amount or a goal or any kind of number you can point us toward as far as your expectations of how big those portfolios are over the next year?
Paul B. Murphy -- Chairman and Chief Executive Officer
Yeah. We're looking at that carefully and we're sort of reevaluating -- let's do Restaurant first. So I would say that the team is doing a review of what's happening in the industry. We're going to have a discussion about where we want it to be. I think our -- we'll probably settle somewhere in the $750 million, $800 million range. Again, I will let the team do their work and bring a recommendation forward and something that we'll consider and decide at that time and obviously consult with our Board about it. So, probably a couple of hundred million dollar reduction over the next two years, I would say, in Restaurant would be a reasonable expectation to be determined.
And then leveraged, that portfolio is a bit more dynamic. In other words, leverage without moderators, if a credit has a covenant violation, it might start off as leverage -- that's not leveraged, but it goes into that bucket. And so it sort of can go bit of a life of its own. It's not like you start with a finite number and it just comes down, but I do expect to see overall leverage borrowers at our Company reduce.
Matt Olney -- Stephens Inc. -- Analyst
Okay. That's helpful. Thank you, guys.
Operator
And ladies and gentlemen, this concludes our question-and-answer session. I'd like to turn the conference back over to Paul for any closing remarks.
Paul B. Murphy -- Chairman and Chief Executive Officer
Okay. So in summary, obviously, our goal is to have a strong regional bank that generates attractive returns for investors and to do that we focus on core growth, really just what we've been talking about and over the last many years, we've been able to generate attractive loan growth and really nice deposit growth, especially last year. So, we have the team in place now that knows how to do that and I would just remind our investors that we're confident in our team and we know that they are extremely committed and determined and like Valerie, I'm looking forward to 2020.
With that, we stand adjourned.
Operator
[Operator Closing Remarks]
Duration: 40 minutes
Call participants:
Paul B. Murphy -- Chairman and Chief Executive Officer
Valerie C. Toalson -- Chief Financial Officer
R.H. Hank Holmes -- Executive Vice President
Samuel M. Tortorici -- President
Brady Gailey -- Keefe, Bruyette & Woods -- Analyst
Jennifer Demba -- SunTrust Robinson Humphrey -- Analyst
Michael Rose -- Raymond James -- Analyst
Brad Milsaps -- Sandler O'Neill + Partners, L.P. -- Analyst
Matt Olney -- Stephens Inc. -- Analyst
